Author(s): Daniel Khoa Le Originally published on Towards AI. Custom LLM agent interacting with external systems (Image by Author) Overview: The Story The Codes Conclusion U+1FAD6 TLDR This article presents an LLM-powered solution to improve the efficiency of ordering and payment processes in food establishments. It involves the integration of an LLM Agent (built with LangChain) which is capable of interacting with API endpoints (e.g. the POS System). This article walks through the codes to build such an agent with practical examples. The codes are published here. You can also interact with the agent here. Demo Video: Notice the signposting/cues agent … please (read more below) in this conversation and how the agent picked only the relevant information to make respective API requests. U+1F956 The Story U+1F9C0 The Inspiration I am a fan of the tasty Käsedings at Zeit für Brot (Berlin, Germany) and I’ve spent quite a bit of time hanging out at their busy Eberswalder spot. It’s where my interest in the tasty breads collides with the reality of waiting in line. I noticed and hypothesized that, to maintain hygiene, cashiers don’t directly handle the food. There would be one cashier and one other staff working in a pair to deal with customers. The cashiers only manage orders and payments, leaving food handling to their peers. U+27A1️ This observation sparked an idea: What if we could make the ordering and payment process more efficient, enhancing customer satisfaction, streamlining staff workflow, and ultimately boosting Zeit für Brot’s revenue (sorry for the buzzwords)? Illustration generated by Author via ChatGPT U+1F4A1 The Solution: The Integration of an LLM-agent This solution proposes integrating a custom LLM agent into the Point of Sale (POS) system. The agent will translate the staff’s orders into API requests, registering items in the POS system without manual entry. U+1F4A1 Workflow Simplified: U+1F469U+1F4BC Staff-Customer Interaction: A staff member receives an order from a customer. He communicates with the LLM agent through his headset. The staff member can engage in conversation with the customer without needing to deactivate the headset, allowing for a seamless interaction. The LLM agent listens to the entire conversation, filtering out irrelevant chatter and focusing on the important details. This is facilitated by the use of specific “agent” and “please” cues (signposting), which enables the LLM agent to distinguish between casual conversation and a direct request for assistance. Example: – Customer A: Do you want to get an espresso and two cinnamon rolls?– Customer B: I don’t know.– Customer A: OK, I’ll decide then. I’ll get a baguette, a cappuccino, and two cheesecakes.– Staff: Agent, add a baguette, a cappuccino, and two cheesecakes, please.– Customer A: Sorry I changed my mind, I won’t get the baguette.– Staff: Agent, remove the baguette, please.– Staff: Agent, proceed to payment by card, please. You can check out the demo here (Image by Author) Request made to the mentioned API endpoints (Image by Author) Under the hood: The agent processes this speech, generating API request(s) to register items to the order. If there’s a change in the order, the staff can proceed with: “Agent, remove one cinnamon roll.” The agent will adjust the order accordingly. Upon finalizing the order, “Agent, proceed to payment with QR codes / cards please” prompts the agent to initiate the payment process, creating a smooth end-to-end transaction. U+1F50C The Codes In this session, I will focus on discussing how to build a custom LLM agent with LangChain. I assume you are familiar to a certain extent with LangChain’s terms and usage. If not, you can checkout their documentation here. I suggest you also refer to my repository when reading the following, as I will not display the entire scripts in this article. Instead, I will highlight the key takeaways for you. What do you need to create a custom LLM agent that can interact with external systems? The tools (that interact with external systems) The prompt (that guides the agent) The agent (and its executor) Now, let’s go through it one by one. U+1F527 The tools I created 3 simple tools to add item(s) to an order remove item(s) from an order proceed to payment with a preferred method These tools are simple functions created with the langchain @tool decorators. You might wonder how the agent knows which tools to take. The answer is that it bases its decisions on the name and the description of the tools. In the following codes: The function’s name add_orders will be considered as the tool’s name. The function’s docstring is treated as the tool’s descriptions. The description of the tool plays an important role because it helps the LLM to reason and decide which tools are relevant in a given context. This add_orders function take an argument order_data in the format of a dictionary (see codes below). The LLM will make sure that the order_data is in the format that we want. You can see that I wrote the docstring as if I am giving instructions to the LLM. The function makes a POST request to an API endpoint. This, ideally, is the system that we want the agent to control. In my case, it is the POS system that the staff uses at the store. @tooldef add_orders(order_data): """Add items to an order by making a POST request with the order_data argument strictly following this format: order_data = [ {'quantity': 2, 'item': 'apple'}, {'quantity': 1, 'item': 'banana'}, {'quantity': 3, 'item': 'cherry'} ] """ # Make a POST request to the FastAPI endpoint url = "http://localhost:8889/add" # Adhere to the expected format of the FastAPI endpoint request_json = {"list_items": order_data} response = requests.post(url, headers={ "Content-Type": "application/json"}, json=request_json) As mentioned previously, besides this add_orders, I have remove_orders and pay_orders functions as well. They form a list of tools at the disposal of the agent. Here is how you do it: # Create an LLM modelllm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)# Create a list of tools for the LLM to picktools = [add_orders, remove_orders, pay_orders]# Bind the tools to the LLM, […]
↧