Recently, I attended a 4-day training course on Agentic AI. It was a profound realization that building an Agent actually requires developers to write code, and even necessitates writing the Tools themselves. Here is a high-level overview of my key takeaways.
1. What is Agentic AI?
Traditional AI chatbots are passive responders. Agentic AI represents a paradigm shift where the AI system is given a goal, the ability to use tools, and the autonomy to take actions to achieve that goal. Instead of just generating text, an Agentic system can perceive, think, act, and observe. It operates in a continuous loop: analyzing the user's intent, planning the steps, executing tools (like APIs or databases), evaluating the results, and deciding if it needs to iterate or provide a final answer.
2. Architecture: Logical vs. Physical
To build a robust Agentic system, we must separate the conceptual design from the actual implementation.
Logical Architecture (The "What")
This layer defines the functional components and their responsibilities, independent of specific vendors.
- Agent Runtime: The core loop where the agent "thinks" and executes code.
- Memory Service: Stores short-term conversation history and long-term user preferences.
- LLM Router: Directs queries to the appropriate model based on cost, speed, or complexity.
- Tool Gateway: The interface managing how the agent connects to external APIs and plugins.
- Registry: A catalog where tools and agent prompts are versioned and discovered.
- Observability: The capability to trace, log, and debug the agent's decision-making process.
Physical Architecture (The "How")
This layer defines the concrete technologies and infrastructure used to bring the logic to life.
- Runtime: Docker & Kubernetes for container orchestration.
- Storage: PostgreSQL (structured data), Redis (caching), Vector DB (semantic search).
- Infrastructure: AWS / Azure / GCP, API Gateways, VPC, and IAM for security.
- Operations: Scaling policies and Monitoring backends (e.g., Prometheus, Datadog).
3. Design Example: Online Travel Planner
Scenario: A user says, "Plan a 5-day trip to Beijing with a budget of 5000 RMB."
Logical Design (Multi-Agent Collaboration)
Instead of one massive prompt, we split the responsibilities into specialized agents orchestrated by a Supervisor:
- Supervisor Agent (The Brain): Parses the intent, extracts parameters (Destination: Beijing, Days: 5, Budget: 5000), and delegates tasks.
- Planner Agent (The Strategist): Uses Map APIs and Weather APIs to create a geographically logical daily itinerary.
- Resource Agent (The Buyer): Uses Airline and Hotel APIs to fetch real-time pricing and availability.
- Budget Agent (The Accountant): Calculates the total cost. If the total exceeds 5000 RMB, it triggers a loop, rejecting the plan and asking the Planner/Resource agents to find cheaper alternatives.
Physical Implementation & Workflow
We use LangGraph to build the state machine that allows the Budget Agent to send tasks backward in the loop. We use MCP (Model Context Protocol) to standardize the connection to various travel APIs (flights, hotels). The system runs in Docker containers managed by Kubernetes, with Redis caching live flight prices to reduce API costs.
| Step | Action | Agent Involved |
|---|---|---|
| 1. Intent Extraction | Identifies destination, duration, and budget constraints. | Supervisor |
| 2. Data Gathering | Fetches flight prices, hotel rates, and weather forecasts. | Resource & Planner |
| 3. Draft Itinerary | Creates a day-by-day plan with routing and pricing. | Planner |
| 4. Budget Audit | Checks if total cost ≤ 5000 RMB. If not, loops back to Step 3. | Budget |
| 5. Final Output | Presents the detailed plan to the user for confirmation. | Supervisor |
Appendix: Day 1 Assignment - Prompt Engineering for Structured Output
Before an Agent can reliably call external tools (like the travel APIs mentioned above), it must first be capable of producing strictly formatted, machine-readable data. This foundational exercise demonstrates how to use JSON Schema to force an LLM to convert messy natural language food orders into a strict JSON structure that a program can execute.
import json
from openai import OpenAI
client = OpenAI()
MODEL = "gpt-4o-mini"
# 1. Define the schema for the model output result
order_schema = {
"type": "object",
"properties": {
"items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"quantity": {"type": "integer"},
"size": {"type": ["string", "null"]},
},
"required": ["name", "quantity", "size"],
"additionalProperties": False,
},
},
},
# to calculate the total items in one order
"total_number": {"type": "integer"},
"required": ["items"],
"additionalProperties": False,
}
# 2. Sample natural language orders
text = [
"2x Americano, 1 large fries and 3 hamburger",
"I'll have a cappuccino and two croissants please",
"order 1 big hamburger and 1 coco cola",
"2x sandwitches and 2 middle sprite",
"3x chicken wings",
]
# 3. Analyze the orders one by one using strict JSON schema
for t in text:
response = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": "Extract every line item from the order text."},
{"role": "user", "content": t},
],
response_format={
"type": "json_schema",
"json_schema": {"name": "Order", "schema": order_schema, "strict": True},
},
)
msg = response.choices[0].message
if msg.refusal:
print("Refused:", msg.refusal)
continue
data = json.loads(msg.content)
# 4. Calculate the count of items programmatically
items = data["items"]
total_number = sum(i["quantity"] for i in items)
# 5. Output results
print("✅", t)
for item in data["items"]:
print(" ", item)
print(f" total_number = {total_number}")
Execution Output:
✅ 2x Americano, 1 large fries and 3 hamburger
{'name': 'Americano', 'quantity': 2, 'size': None}
{'name': 'fries', 'quantity': 1, 'size': 'large'}
{'name': 'hamburger', 'quantity': 3, 'size': None}
total_number = 6
✅ I'll have a cappuccino and two croissants please
{'name': 'cappuccino', 'quantity': 1, 'size': None}
{'name': 'croissant', 'quantity': 2, 'size': None}
total_number = 3
✅ order 1 big hamburger and 1 coco cola
{'name': 'big hamburger', 'quantity': 1, 'size': None}
{'name': 'coco cola', 'quantity': 1, 'size': None}
total_number = 2
✅ 2x sandwitches and 2 middle sprite
{'name': 'sandwitches', 'quantity': 2, 'size': None}
{'name': 'sprite', 'quantity': 2, 'size': 'middle'}
total_number = 4
✅ 3x chicken wings
{'name': 'chicken wings', 'quantity': 3, 'size': None}
total_number = 3Why this matters for Agentic AI: You cannot build a reliable Agent on top of unstructured text. By enforcing a json_schema with strict: True, we guarantee that the LLM's output can be directly parsed by our Python code. This same mechanism is exactly how LLMs reliably pass parameters to external Tools (e.g., passing {"destination": "Beijing", "days": 5} to a flight search API).
No comments:
Post a Comment