# Week 10: Multi-agent workflows Applied AI Course — Sawan Ruparel Public self-study edition. See https://applied-ai.sawanruparel.com/updates/ for version notes. # Multi-Agent Workflows ## From Solo Agents to Agent Teams Week 10 — Applied AI Course, Spring 2026 --- *When one agent isn't enough, build a team.* --- # Why Single Agents Hit Limits ## Context Window Constraints - Complex tasks require more context than a single agent can hold effectively - Tool outputs, conversation history, and documents compete for token budget - Performance degrades as context grows — the "lost in the middle" problem ## Role Confusion - One agent asked to be researcher, writer, critic, and coder simultaneously - System prompts become sprawling instruction manuals (2000+ words) - The agent switches "hats" mid-task and loses focus on each role ## Error Cascading - A single mistake early in a long chain corrupts everything downstream - No second opinion — the agent cannot catch its own reasoning errors - Retrying the whole pipeline is expensive; isolating the failure is hard ## The Single-Agent Ceiling - Beyond a certain task complexity, adding more instructions yields diminishing returns - You need **specialization**, not a longer prompt ## Sources - [Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023)](https://arxiv.org/abs/2307.03172) - [Building Effective Agents (Anthropic, 2024)](https://www.anthropic.com/research/building-effective-agents) --- # Specialized Agents > One Generalist Agent ## Single Generalist Agent - One massive system prompt covering all responsibilities - All tools loaded into one context (20+ tools) - Agent must decide what "mode" it's in at every step - Context window fills up fast with mixed concerns - Hard to debug — which part of the prompt caused the error? - Changing one capability risks breaking others - Testing requires end-to-end runs every time ## Multi-Agent Team - Each agent has a focused role and concise instructions - Agents carry only the tools they need (3-5 tools each) - Clear separation of concerns — researcher researches, writer writes - Each agent gets a fresh, focused context window - Failures are isolated to a single agent and easier to trace - Swap or upgrade one agent without touching the rest - Unit test each agent independently, integration test the team --- **The key insight:** Multi-agent is not about having more LLM calls. It's about giving each call a smaller, clearer job. --- # Multi-Agent Framework Landscape ## AutoGen → Microsoft Agent Framework (Microsoft) - Conversation-driven multi-agent framework; **AutoGen v1.0 GA** ships the v2 API as the default - Agents communicate through chat-like message passing - Strong support for human-in-the-loop workflows; GroupChat abstraction for multi-party agent conversations - Microsoft has **merged AutoGen with Semantic Kernel into the Microsoft Agent Framework (MAF)**, targeting enterprise .NET with production middleware and observability ## CrewAI - Role-based agent teams with a "crew" metaphor - Declarative: define agents, tasks, and processes in config - Now ships **enterprise observability and scheduling**; powers **~2 billion agentic executions** and is used by **60%+ of the Fortune 500** - **CrewAI Flows** is the recommended architecture for production multi-agent pipelines ## LangGraph (LangChain) - Graph-based orchestration with explicit state machines - Nodes are agents or functions, edges are conditional transitions - Fine-grained control over routing, loops, and error handling; best for complex workflows where you need deterministic control flow - **v0.4 (April 2026)** adds redesigned HITL checkpoints that surface automatically in `.invoke()` responses, durable Postgres/Redis state persistence, and improved graph-based audit trails; it now leads CrewAI in GitHub stars ## Claude Agent SDK (Anthropic) - Lightweight Python SDK for building agent loops with Claude - Built-in support for tool use, handoffs between agents, and guardrails - First-class MCP integration for connecting agents to external services - Designed for production: tracing, streaming, structured output ## How to Choose - **Graph-shaped, stateful workflows with audit needs?** LangGraph - **Declarative role-based crews?** CrewAI - **.NET enterprise or conversational GroupChat?** AutoGen / Microsoft Agent Framework - **Production with Claude?** Claude Agent SDK - **Prototyping quickly?** CrewAI for declarative crews, AutoGen for flexible conversation patterns ## Sources - [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (Wu et al., 2023)](https://arxiv.org/abs/2308.08155) - [AutoGen GitHub Repository (Microsoft)](https://github.com/microsoft/autogen) - [CrewAI GitHub Repository](https://github.com/crewAIInc/crewAI) - [LangGraph Documentation (LangChain)](https://langchain-ai.github.io/langgraph/) - [Claude Agent SDK (Anthropic)](https://github.com/anthropics/claude-agent-sdk-python) - [LangGraph v0.4: HITL checkpoints & state persistence — AI Tech Connect](https://aitechconnect.in/news/langgraph-v04-hitl-checkpoints-state-persistence) - [AI agent frameworks compared (AutoGen v1.0, MAF) — PE Collective](https://pecollective.com/blog/ai-agent-frameworks-compared/) - [Multi-agent AI in 2026: CrewAI, LangGraph, AutoGen — dev.to](https://dev.to/ottoaria/multi-agent-ai-in-2026-build-production-systems-with-crewai-langgraph-autogen-5e40) - [LangGraph vs CrewAI vs AutoGen: which framework for your enterprise in 2026 — Towards AI](https://pub.towardsai.net/langgraph-vs-crewai-vs-autogen-which-ai-agent-framework-should-your-enterprise-use-in-2026-3a9ebb407b09) --- # Pattern 1: Sequential Pipeline Agent A's output becomes Agent B's input, forming a linear chain. ```mermaid flowchart LR R[Researcher] -->|raw data| A[Analyst] A -->|key findings| W[Writer] W -->|draft| E[Editor] E --> F[Final report] ``` ## When to Use - Tasks that naturally decompose into ordered stages - Each stage has a clearly different skill requirement - Output format of stage N matches input format of stage N+1 ## Strengths - Simple to reason about and debug - Easy to swap out individual stages - Each agent gets a focused context ## Weaknesses - Slow — total latency is sum of all stages - Errors in early stages propagate forward - No feedback loops — agent C can't ask agent A for clarification --- # Pattern 2: Hierarchical — Manager Delegates to Workers A manager agent breaks down tasks and delegates to specialized workers. ```mermaid flowchart TB M["Manager
(decompose · delegate · synthesize)"] M -->|sub-task| Wa["Coder
file_edit, terminal"] M -->|sub-task| Wb["Researcher
web_search, read_url"] M -->|sub-task| Wc["Tester
run_tests, lint"] Wa -->|result| M Wb -->|result| M Wc -->|result| M M --> Out[Synthesized output] ``` ## Key Design Decisions - **Manager has NO tools** — it only plans, delegates, and synthesizes - **Workers have NO planning ability** — they execute a specific sub-task - Manager decides when the task is done or needs another iteration - Workers report results back to the manager, not to each other ## When to Use - Tasks requiring multiple distinct skills - You want centralized control over workflow orchestration - The manager can effectively decompose the problem --- # Pattern 3: Collaborative — Peer Agents Discuss and Iterate Agents communicate as peers, refining each other's work through rounds of feedback. ```mermaid sequenceDiagram participant P as Proposer participant C as Critic P->>C: Draft v1 C->>P: Issues: missing X, wrong format P->>C: Draft v2 (addresses feedback) C->>P: Better — consider edge case P->>C: Draft v3 C-->>P: Approved (or max_rounds hit) ``` ## How It Works - Agents take turns producing and critiquing output - Each round uses the previous round's feedback as input - Terminates when critic approves, max rounds hit, or quality threshold met ## Strengths - Produces higher-quality output through iterative refinement - Natural fit for writing, code review, and design tasks - Simple two-agent setup can be very effective ## Watch Out For - Agents can get stuck in "polite loops" — always finding something to critique - Set a **max iteration count** (typically 3-5 rounds) - Define clear acceptance criteria so the critic knows when to stop --- # Pattern 4: Competitive — Multiple Proposals, Best Wins Multiple agents independently solve the same problem; a judge selects the best result. ```mermaid flowchart TB P[Shared prompt] --> A["Agent A
(conservative)"] P --> B["Agent B
(creative)"] P --> C["Agent C
(analytical)"] A --> J{"Judge
correctness · quality"} B --> J C --> J J --> W[Winner or merged best parts] ``` ## When to Use - Problems with multiple valid approaches (creative tasks, architecture decisions) - High-stakes outputs where diversity of thought reduces risk - You can afford the extra compute cost (N parallel agent calls) ## Trade-offs - **Pro:** Reduces variance — unlikely all agents make the same mistake - **Pro:** Judge can merge the best parts of multiple proposals - **Con:** N times the cost of a single agent - **Con:** Judge must be capable of evaluating quality — non-trivial for complex tasks --- # The Supervisor Pattern A router agent dynamically selects which worker to invoke based on the current state. ```mermaid flowchart TB S{"Supervisor
reads state, picks next"} State[(Shared state)] S -->|needs_data| R[Researcher] S -->|needs_code| Co[Coder] S -->|needs_review| Rv[Reviewer] S -->|done| Fin([FINISH]) R --> State Co --> State Rv --> State State --> S ``` ## How It Differs from Hierarchical - **Hierarchical:** Manager plans all sub-tasks upfront, then delegates - **Supervisor:** Router makes one decision at a time based on current state - Supervisor is reactive; hierarchical is proactive ## Implementation with LangGraph - Model the supervisor as a node that returns the name of the next node - Each worker is a node that updates shared state - Conditional edges route from supervisor to the selected worker - After each worker completes, control returns to the supervisor ## Key Design Choice The supervisor agent should use **structured output** (e.g., return `{"next": "coder", "task": "..."}`) rather than free-form text to make routing deterministic and parseable. ## Sources - [LangGraph Documentation (LangChain)](https://langchain-ai.github.io/langgraph/) --- # Swarm Pattern — Dynamic Agent Handoffs Agents hand off control to each other based on conversation context. No central coordinator. ```mermaid sequenceDiagram participant U as User participant T as Triage participant Tr as Travel participant E as Expense U->>T: "Book a flight, then file expenses" T->>Tr: handoff(travel_agent, history) Tr->>U: "Flight booked" Tr->>E: handoff(expense_agent, history) E->>U: "Expense report filed" Note over U,E: Conversation history travels with the user ``` ## Key Properties - **No central router** — each agent decides when to hand off and to whom - **Conversation history travels** with the user across handoffs - Each agent declares which other agents it can hand off to - Handoff is implemented as a tool call: `handoff(agent_name, context)` ## The Claude Agent SDK Approach - Agents define `handoffs` — a list of agents they can transfer to - When an agent calls a handoff tool, the SDK swaps the active agent - The new agent receives the full conversation so far - Natural for customer-service-style flows where the user drives the conversation ## When to Use - Conversational systems where the user's needs shift mid-session - Each agent "owns" a domain (billing, tech support, sales) - You want the system to feel like one seamless conversation ## Sources - [Claude Agent SDK (Anthropic)](https://github.com/anthropics/claude-agent-sdk-python) - [OpenAI Swarm (Experimental)](https://github.com/openai/swarm) --- # Choosing the Right Pattern for Your Use Case ## Step 1: Do You Actually Need Multi-Agent? - Can a single agent with good tools solve this? **Start there.** - Multi-agent adds latency, cost, and debugging complexity - Rule of thumb: if your system prompt is under 500 words and you have fewer than 8 tools, a single agent is probably fine ## Step 2: Is the Workflow Fixed or Dynamic? - **Fixed order** (A then B then C): use **Sequential Pipeline** - **Dynamic routing** (depends on intermediate results): use **Supervisor** or **Swarm** - **User-driven conversation** with shifting topics: use **Swarm** ## Step 3: How Many Distinct Roles Are Needed? - **2 roles** (writer + critic): use **Collaborative** pattern - **3-5 specialized roles** with a clear coordinator: use **Hierarchical** - **Many interchangeable workers**: use **Supervisor** with a worker pool ## Step 4: How Important Is Output Quality vs. Speed? - **Quality is paramount** and cost is acceptable: use **Competitive** (multiple proposals + judge) - **Speed matters** and tasks are independent: use **Parallel workers** under a manager - **Iterative refinement** is the goal: use **Collaborative** with max iterations ## Step 5: Summary Decision Matrix | Scenario | Pattern | Why | |---|---|---| | ETL pipeline | Sequential | Fixed stages, clear data flow | | Software project | Hierarchical | Manager decomposes, workers build | | Blog post writing | Collaborative | Writer + editor iterate to quality | | Architecture design | Competitive | Multiple proposals reduce risk | | Customer support | Swarm | User needs shift dynamically | | Complex research | Supervisor | Router picks next step based on findings | --- # Designing Individual Agents Every agent in a multi-agent system needs four things defined clearly. ## 1. Role — Who Is This Agent? - A one-sentence identity: "You are a senior code reviewer focused on security vulnerabilities" - Defines the agent's perspective, expertise, and voice - Constrains the agent to stay in its lane — prevents role drift ## 2. Tools — What Can This Agent Do? - Give each agent **only** the tools it needs (principle of least privilege) - A researcher gets `web_search` and `read_url`, not `file_write` - A coder gets `file_edit` and `run_command`, not `send_email` - Fewer tools = fewer distractions = better tool selection ## 3. Instructions — How Should It Behave? - Specific output format expectations (JSON, markdown, structured report) - Quality criteria: "Flag any function longer than 50 lines" - Boundary rules: "If you cannot complete the task, return a structured error, do not guess" ## 4. Constraints — What Are the Guardrails? - Max iterations or token budget per agent - Forbidden actions (e.g., "never execute DELETE queries") - Escalation rules: "If confidence is below 70%, hand off to a human" ## The Agent Card Pattern Define each agent as a config object or YAML file — this makes agents composable, testable, and version-controlled: ```yaml name: security_reviewer role: "Senior security engineer reviewing code for vulnerabilities" model: claude-sonnet-4-20250514 tools: [read_file, grep_codebase, static_analysis] max_tokens: 4096 instructions: | Review the provided code for OWASP Top 10 vulnerabilities. Output a structured JSON report with severity ratings. constraints: max_iterations: 3 forbidden_actions: [file_write, execute_code] ``` --- # Inter-Agent Communication ## Message Passing - Agents send structured messages directly to each other - Each message includes sender, recipient, content, and metadata - Natural fit for conversational patterns (AutoGen, Claude Agent SDK) - **Pro:** Clear audit trail of who said what - **Con:** Message history can grow large across many rounds - Example: Agent A sends `{role: "researcher", content: "Found 3 relevant papers..."}` to Agent B ## Shared State (Blackboard) - All agents read from and write to a central state object - State is typically a dictionary or typed schema - Natural fit for LangGraph (state is passed through graph edges) - **Pro:** Agents don't need to know about each other — just the state - **Con:** Race conditions if agents run in parallel without locks - Example: `state["research_findings"] = [...]` written by researcher, read by writer ## Event-Driven (Pub/Sub) - Agents publish events; other agents subscribe to event types - Loose coupling — agents don't know who's listening - Good for systems where new agents are added frequently - **Pro:** Highly extensible; add new agents without changing existing ones - **Con:** Harder to debug — event flows are implicit, not explicit - Example: Researcher publishes `DataReady` event; Analyzer subscribes and activates ## Tool-as-Communication - One agent invokes another agent as if it were a tool - The "inner agent" runs and returns structured output - Simplest pattern — no message bus or shared state needed - **Pro:** Familiar tool-use interface; easy to implement - **Con:** Only supports request-response, not multi-turn dialogue - Example: Manager calls `run_agent(name="researcher", query="Find recent papers on X")` ## Sources - [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (Wu et al., 2023)](https://arxiv.org/abs/2308.08155) - [LangGraph Documentation (LangChain)](https://langchain-ai.github.io/langgraph/) - [Claude Agent SDK (Anthropic)](https://github.com/anthropics/claude-agent-sdk-python) --- # State Management: Sharing Context Without Losing Coherence ## The Core Problem - Agent A discovers important context that Agent B needs later - But Agent B has a fresh context window — it doesn't know what Agent A learned - Passing the entire conversation history bloats context and confuses agents ## Strategy 1: Structured State Object - Define a typed schema for shared state (e.g., a Pydantic model) - Each agent reads relevant fields and writes back its outputs - The state object is the single source of truth ```python class PipelineState(BaseModel): query: str research_notes: list[str] = [] draft: str = "" review_feedback: str = "" final_output: str = "" status: Literal["researching", "drafting", "reviewing", "done"] ``` ## Strategy 2: Summary Handoffs - When Agent A finishes, it produces a **structured summary** of its work - Agent B receives only the summary, not Agent A's full conversation - Keeps each agent's context small and focused - Trade-off: some nuance is lost in summarization ## Strategy 3: Artifact Store - Agents write intermediate outputs to a shared file system or database - Other agents read artifacts by reference (file path, document ID) - Works well for large outputs (code files, datasets, reports) - Natural for code generation pipelines where agents produce actual files ## Anti-Patterns to Avoid - **Dumping full conversation history** into the next agent's context — causes confusion - **Relying on implicit state** ("the last agent probably did X") — be explicit - **Mutable state without versioning** — you can't debug if you don't know what changed when - **No state validation** — agents should validate that required fields exist before proceeding --- # Tool Sharing vs. Tool Isolation ## Share Tools When... - Multiple agents need to read from the same data source - Tools are stateless and safe to call concurrently (e.g., `web_search`) - You want to reduce configuration duplication - The tool is a read-only query (database lookups, API reads) - Example: Both researcher and fact-checker share the `web_search` tool **Shared tool architecture:** ```mermaid flowchart TB A[Agent A] --> T[Shared tool: web_search] B[Agent B] --> T ``` ## Isolate Tools When... - The tool has side effects (file writes, API mutations, sending messages) - Agents operate on different resources and shouldn't see each other's work - You need to enforce permissions (coder can write code, reviewer cannot) - The tool maintains state between calls (database connections, sessions) - Example: Only the coder agent gets `file_write`; the reviewer only gets `file_read` **Isolated tool architecture:** ```mermaid flowchart TB A[Agent A — coder] --> AT["file_write
terminal"] B[Agent B — reviewer] --> BT["file_read
lint"] ``` --- **Rule of thumb:** Read-only tools can be shared. Write tools should be isolated to the agent responsible for that action. --- # Error Recovery: What Happens When One Agent Fails? ## Types of Agent Failures 1. **Tool failure** — an API call times out, a file doesn't exist, a command returns an error 2. **Quality failure** — the agent produces output that doesn't meet requirements 3. **Loop failure** — the agent gets stuck repeating the same action 4. **Budget failure** — the agent exhausts its token or iteration limit ## Strategy 1: Retry with Feedback - Catch the error, include it in the agent's next prompt - "Your previous attempt failed with: [error]. Try a different approach." - Set a max retry count (2-3 retries) to avoid infinite loops - Works well for transient tool failures ## Strategy 2: Fallback Agent - If the primary agent fails, route to a backup agent with different instructions - Example: if the code-generation agent produces code that fails tests, hand off to a "fix-it" agent that specializes in debugging - The fallback agent receives the original task + the error context ## Strategy 3: Supervisor Rerouting - The supervisor detects the failure and reassigns the task - May choose a different worker, simplify the task, or break it into smaller pieces - Requires the supervisor to understand what went wrong (structured error reports help) ## Strategy 4: Graceful Degradation - Return a partial result with a clear indication of what's missing - "Research completed for 3 of 5 topics. Topics 4 and 5 failed due to API rate limits." - Let the human decide whether to retry or accept the partial result ## Implementation Checklist - Every agent should return structured status: `{success: bool, result: ..., error: ...}` - Log all failures with context for debugging - Set timeouts on every agent invocation — never wait forever - Design the orchestrator to handle failures, not just the happy path --- # Orchestration Engines ## LangGraph: State Machine Orchestration - Define your workflow as a **directed graph** with nodes and edges - Nodes are agents or functions; edges define transitions - **Conditional edges** let you route based on state (if/else at graph level) - Built-in support for cycles (agent loops), checkpointing, and human-in-the-loop - As of **v0.4 (April 2026)**, HITL checkpoints surface automatically in `.invoke()` responses and state persists durably in Postgres/Redis, giving graph-based audit trails out of the box - Best for: complex workflows with branching logic and loops ```python graph = StateGraph(PipelineState) graph.add_node("researcher", researcher_agent) graph.add_node("writer", writer_agent) graph.add_node("reviewer", reviewer_agent) graph.add_edge("researcher", "writer") graph.add_conditional_edges("reviewer", route_after_review, {"approved": END, "needs_revision": "writer"}) graph.set_entry_point("researcher") ``` ## Event-Driven Architecture - Agents subscribe to events and react independently - A message broker (Redis, Kafka, or even an in-memory queue) delivers events - Best for: systems where agents need to react to real-time triggers - More infrastructure overhead but highly scalable ## Simple Function Orchestration - For straightforward pipelines, just call agents in sequence in Python - No framework needed — `result_a = agent_a(input); result_b = agent_b(result_a)` - Best for: pipelines with 2-4 stages and no branching - Don't reach for a framework until you need conditional routing or loops ## When to Use What | Complexity | Approach | Example | |---|---|---| | Linear, 2-4 steps | Plain Python functions | Research then write | | Branching, loops | LangGraph | Code review with revision cycles | | Real-time, many agents | Event-driven | Customer support with live routing | | Conversational handoffs | Claude Agent SDK | Multi-department chatbot | ## Sources - [LangGraph Documentation (LangChain)](https://langchain-ai.github.io/langgraph/) - [LangGraph v0.4: HITL checkpoints & state persistence — AI Tech Connect](https://aitechconnect.in/news/langgraph-v04-hitl-checkpoints-state-persistence) - [Claude Agent SDK (Anthropic)](https://github.com/anthropics/claude-agent-sdk-python) --- # MCP in Multi-Agent Systems Each agent connects to its own set of MCP servers, giving it access to domain-specific tools. ```mermaid flowchart TB subgraph Orch[Orchestrator] R[Research agent] C[Coding agent] D[DevOps agent] end R --> RM["Brave · Arxiv · Wikipedia"] C --> CM["GitHub · Filesystem · LSP"] D --> DM["AWS · Docker · Kubernetes"] ``` ## Why MCP Fits Multi-Agent Naturally - MCP provides a **standard protocol** for tool discovery and invocation - Each agent can spin up its own MCP server connections independently - Agents get tool descriptions automatically — no manual tool schema wiring - Adding new capabilities = adding a new MCP server, not changing agent code ## Architecture Pattern: Agent + MCP Config ```yaml agents: researcher: model: claude-sonnet-4-20250514 mcp_servers: - brave-search - arxiv-reader coder: model: claude-sonnet-4-20250514 mcp_servers: - github - filesystem - terminal devops: model: claude-sonnet-4-20250514 mcp_servers: - aws-cli - docker ``` ## Key Benefit Each agent only "sees" the tools from its MCP servers. The researcher cannot accidentally run shell commands. The coder cannot accidentally search the web. **Tool isolation through MCP server assignment.** ## Sources - [Model Context Protocol Specification (Anthropic)](https://modelcontextprotocol.io/specification/2025-11-25) - [Introducing the Model Context Protocol (Anthropic, 2024)](https://www.anthropic.com/news/model-context-protocol) --- # Example: Automated Code Review Pipeline ```mermaid flowchart LR PR[Pull request diff] --> P[Planner] P -->|review tasks| R1[Reviewer · file 1] P --> R2[Reviewer · file 2] P --> R3[Reviewer · file N] P --> T[Tester] R1 --> Rep[Reporter] R2 --> Rep R3 --> Rep T --> Rep Rep --> GH[GitHub PR comment] ``` ## Step 1: Planner Agent - **Input:** Pull request diff and description - **Action:** Analyzes the scope of changes and creates a review plan - **Output:** List of review tasks: "Check error handling in auth.py", "Verify SQL query safety in db.py", "Review test coverage for new endpoints" - **Tools:** `github_pr_diff`, `file_read` ## Step 2: Reviewer Agent (runs in parallel per file) - **Input:** One file diff + review instructions from planner - **Action:** Reviews code for bugs, style issues, security concerns, and performance - **Output:** Structured review comments with severity (critical / warning / suggestion) - **Tools:** `file_read`, `grep_codebase` (to check related code) ## Step 3: Tester Agent - **Input:** The changed files and existing test suite - **Action:** Runs existing tests, checks coverage, suggests missing test cases - **Output:** Test results + list of recommended new tests - **Tools:** `run_tests`, `coverage_report` ## Step 4: Reporter Agent - **Input:** All reviewer comments + test results - **Action:** Synthesizes findings into a single coherent review - **Output:** PR comment with organized feedback, prioritized by severity - **Tools:** `github_post_comment` --- **Why multi-agent here?** A single agent reviewing a large PR would run out of context. Parallelizing reviewers across files keeps each one focused and fast. The reporter ensures the human sees one clean summary, not four separate dumps. --- # Example: Research Agent Team ```mermaid flowchart LR Q[Research question] --> S[Searcher] S -->|sources| A[Analyzer] A -->|structured analysis| W[Writer] W -->|draft brief| F[Fact-checker] F -->|verified brief| Out[Final report] F -.->|re-verifies independently| Q ``` ## Step 1: Searcher Agent - **Input:** Research question from user - **Action:** Breaks question into sub-queries, searches multiple sources - **Output:** Collection of relevant sources with summaries and URLs - **Tools:** `web_search`, `arxiv_search`, `read_url` - **Constraint:** Must find at least 5 sources from 2+ different domains ## Step 2: Analyzer Agent - **Input:** Collected sources and original research question - **Action:** Identifies key themes, contradictions, consensus points, and gaps - **Output:** Structured analysis with claims mapped to supporting sources - **Tools:** `read_url` (for deep reading of top sources) - **Constraint:** Every claim must cite at least one source ## Step 3: Writer Agent - **Input:** Structured analysis + original question - **Action:** Produces a well-organized research brief with sections, citations, and a summary - **Output:** Markdown document with executive summary, findings, and source list - **Tools:** None (pure generation task) - **Constraint:** Must follow provided outline template ## Step 4: Fact-Checker Agent - **Input:** Written brief + original sources - **Action:** Verifies each claim against cited sources, flags unsupported statements - **Output:** Annotated version with confidence scores and any corrections - **Tools:** `web_search`, `read_url` (independent verification) - **Constraint:** Claims not supported by sources must be flagged or removed --- **Key insight:** The fact-checker works independently from the searcher. It re-verifies claims rather than trusting the searcher's summaries, catching errors that propagated through the pipeline. --- # Example: Customer Support with Agent Routing ```mermaid flowchart TB U["Customer:
'I was charged twice'"] --> T{Triage agent} T -->|billing| B["Billing agent
lookup_charges
process_refund"] T -->|technical| Tech[Tech support agent] T -->|account| Acc[Account agent] B --> R[Resolution to user] B -->|low confidence or
angry customer| E["Escalation agent
(human ticket)"] ``` ## Design Decisions - **Triage is cheap:** uses a fast model (Haiku) for classification only - **Specialists are thorough:** use a capable model (Sonnet) with domain tools - **Escalation is a first-class agent**, not an afterthought — it knows how to summarize the conversation for a human agent - Each specialist has access to **different backend systems** via MCP ## Handoff Mechanics (Swarm Pattern) - Triage agent uses `handoff(billing_agent)` with conversation context - Billing agent can `handoff(escalation_agent)` if it detects it cannot resolve - Customer experiences one seamless conversation across all agents --- # Example: Data Analysis Pipeline ```mermaid flowchart LR Src[CSV / DB / API] --> I["Ingestion
schema + profile"] I -->|clean data| A["Analysis
Python sandbox"] A -->|results| V["Visualization
matplotlib / plotly"] A --> N[Narrative] V --> N N --> R[Markdown report] ``` ## Step 1: Ingestion Agent - **Input:** User query + data source reference (CSV file, database, API endpoint) - **Action:** Loads data, inspects schema, identifies data quality issues - **Output:** Clean dataframe + data profile (column types, null counts, distributions) - **Tools:** `read_csv`, `query_database`, `pandas_profile` - **Key behavior:** Flags data quality issues before analysis begins — missing values, type mismatches, outliers ## Step 2: Analysis Agent - **Input:** Clean data + user's analytical question - **Action:** Writes and executes Python/SQL analysis code in a sandbox - **Output:** Statistical results, computed metrics, identified patterns - **Tools:** `execute_python` (sandboxed), `query_database` - **Key behavior:** Iterates on its own code — if execution fails, it reads the error and fixes the code (up to 3 retries) ## Step 3: Visualization Agent - **Input:** Analysis results + data + user's original question - **Action:** Selects appropriate chart types, generates publication-quality visualizations - **Output:** Chart images (PNG/SVG) + alt-text descriptions - **Tools:** `execute_python` (matplotlib/plotly), `save_image` - **Key behavior:** Chooses chart type based on data shape — bar charts for comparisons, line charts for trends, scatter for correlations ## Step 4: Narrative Agent - **Input:** Analysis results + visualizations + original question - **Action:** Writes a plain-English summary of findings with embedded charts - **Output:** Markdown report with key insights, methodology notes, and caveats - **Tools:** None (pure generation from structured inputs) - **Key behavior:** Highlights surprising findings and explicitly states limitations --- **Why not one agent?** The analysis agent needs a code execution sandbox. The visualization agent needs different libraries. The narrative agent needs writing skill, not coding. Separation lets each use the right model temperature and tool set. --- # The Debate Pattern Agents argue different positions to improve output quality through structured disagreement. ```mermaid sequenceDiagram participant A as Advocate participant D as Devil's Advocate participant J as Judge A->>D: Position v1 D->>A: Counter-arguments A->>D: Position v2 (defended) D->>A: New counter-arguments A->>J: Final position + history D->>J: Final critique + history J-->>A: Synthesized verdict ``` ## How It Works 1. **Agent A** (Advocate) presents a position or solution 2. **Agent B** (Devil's Advocate) argues against it, finding weaknesses 3. **Agent A** responds to criticisms and strengthens its argument 4. After N rounds, a **Judge agent** synthesizes the strongest points from both sides ## Why Debate Improves Quality - Forces exploration of alternative perspectives that a single agent would miss - Surfaces hidden assumptions — the devil's advocate questions things the advocate takes for granted - Reduces overconfidence — the advocate must defend its claims with evidence - The final synthesis is more nuanced than either position alone ## Practical Implementation ```python def debate(question, rounds=3): advocate_history = [] critic_history = [] for round in range(rounds): # Advocate presents/defends position advocate_msg = advocate_agent.run( question=question, criticism=critic_history[-1] if critic_history else None ) advocate_history.append(advocate_msg) # Critic challenges the position critic_msg = critic_agent.run( position=advocate_msg, round=round ) critic_history.append(critic_msg) # Judge synthesizes return judge_agent.run( question=question, debate_history=list(zip(advocate_history, critic_history)) ) ``` ## Good Use Cases - **Architecture decisions** — "Should we use microservices or a monolith?" - **Risk assessment** — one agent identifies risks, the other argues they're manageable - **Content quality** — advocate writes, critic finds logical gaps - **Code design** — one proposes an approach, the other stress-tests edge cases ## When to Skip It - Simple factual questions (no room for debate) - Time-critical tasks (debate adds 3-5x latency) - Tasks where there's a clear correct answer (use validation instead) --- # Self-Reflection: An Agent Critiques Its Own Output ```mermaid flowchart LR Task[Task] --> G["Generator
'write the best output'"] G --> Draft[Draft] Draft --> C["Critic
'find everything wrong'"] C --> FB[Critique] FB --> G2[Generator revises] G2 --> Final[Final output] ``` ## The Simplest Multi-Agent Pattern - Same model, two passes: **generate** then **critique** - The critic pass has a different system prompt focused on finding problems - Technically one model, but two distinct "agents" with different roles - Surprisingly effective — often catches 30-50% of errors in the first draft ## How It Works ```python # Pass 1: Generate draft = generator_agent.run( "Write a function to parse CSV files with error handling" ) # Pass 2: Reflect critique = critic_agent.run(f""" Review this code for: 1. Edge cases not handled 2. Security vulnerabilities 3. Performance issues 4. Missing error handling Code to review: {draft} """) # Pass 3: Revise (generator sees its own work + critique) final = generator_agent.run(f""" Original code: {draft} Review feedback: {critique} Please revise the code addressing all feedback. """) ``` ## Why Separate Prompts Matter - The generator prompt says: "Write the best code you can" - The critic prompt says: "Find everything wrong with this code" - These are fundamentally **different cognitive tasks** — separating them improves both - A single prompt saying "write code and check it" produces weaker self-critique ## Reflection Checklist Templates Provide the critic with a structured checklist rather than open-ended critique: - **Code:** Error handling, edge cases, security, performance, readability - **Writing:** Accuracy, completeness, tone, audience appropriateness, logical flow - **Analysis:** Data validity, methodology, statistical significance, alternative explanations ## Limitations - The critic has the same knowledge gaps as the generator (same model) - Will not catch errors that require external information (e.g., wrong API syntax) - Adds one extra LLM call per task — evaluate if the quality gain justifies the cost - For critical tasks, pair self-reflection with external validation (tests, linters, fact-checking tools) ## Sources - [Self-Refine: Iterative Refinement with Self-Feedback (Madaan et al., 2023)](https://arxiv.org/abs/2303.17651) - [Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023)](https://arxiv.org/abs/2303.11366) --- # Cost Control: Agent Loops Can Be Expensive ## The Cost Multiplication Problem - A single agent call: ~$0.01-0.05 (depending on model and context size) - A 4-agent pipeline: ~$0.04-0.20 per run - A 4-agent pipeline with 3 revision loops: ~$0.12-0.60 per run - A competitive pattern with 3 proposals + judge: ~$0.04-0.20 per decision - **At scale, these costs multiply fast** — 1000 requests/day = $120-600/day ## Budget Enforcement Strategies ### Per-Agent Token Limits - Set `max_tokens` on each agent call to prevent runaway generation - A researcher doesn't need 8000 tokens to summarize findings — cap it at 2000 - This also forces agents to be concise ### Max Iteration Caps - Every loop needs a hard stop: `max_iterations = 5` - Collaborative patterns are the biggest risk — agents can "refine" forever - Log a warning when an agent hits its iteration cap ### Early Stopping Conditions - Define "good enough" criteria and stop iterating when met - Example: "If the test pass rate is above 95%, stop the review cycle" - Don't let perfect be the enemy of good (or cheap) ### Model Tiering & Multi-Model Routing - Use cheaper models for simple tasks (classification, routing, formatting) - Reserve expensive models for tasks that need reasoning (analysis, code generation) - Example: Triage agent uses Haiku ($0.25/M tokens), specialist uses Sonnet ($3/M tokens) - A production pattern in 2026 is **multi-model routing**: the orchestrator dynamically routes each subtask across providers — **GPT-5.4, Claude Opus 4.7, Gemini 3.1 Pro** — choosing based on cost, latency, and capability rather than committing the whole pipeline to one model ## Cost Monitoring Dashboard Track these metrics per agent and per workflow: - **Tokens consumed** (input + output) per agent per run - **Number of iterations** before convergence - **Cost per completed workflow** (sum of all agent calls) - **Abort rate** (how often workflows hit budget limits) - Set alerts when cost per workflow exceeds 2x the expected baseline ## Sources - [Building smarter AI agents with multi-model routing in 2026 — AIThority](https://aithority.com/machine-learning/from-gpt-5-5-to-deepseek-v4-how-developers-are-building-smarter-ai-agents-with-multi-model-routing-in-2026/) --- # Observability: Tracing Multi-Agent Conversations ## Why Multi-Agent Systems Are Hard to Debug - A 4-agent pipeline produces 4+ LLM calls, each with its own input/output - When the final output is wrong, which agent introduced the error? - Agent interactions create emergent behavior — the bug may not be in any single agent - Without tracing, debugging is guesswork ## What to Trace ### Per-Agent Traces - **Input:** What prompt and context did the agent receive? - **Output:** What did the agent produce? - **Tool calls:** Which tools were called, with what arguments, and what was returned? - **Token usage:** Input tokens, output tokens, total cost - **Latency:** How long did this agent take? - **Decision points:** Why did the supervisor choose this agent? ### Workflow-Level Traces - **Full execution graph:** Which agents ran, in what order, with what data flow - **State transitions:** How did shared state change after each agent - **Iteration count:** How many loops before convergence - **Total cost and latency** for the complete workflow ## Tracing Tools - **LangSmith** — First-party tracing for LangGraph and LangChain workflows - **Braintrust** — Model evaluation and tracing with multi-step support - **Arize Phoenix** — Open-source observability for LLM applications - **OpenTelemetry** — Standard tracing protocol; some LLM frameworks emit OTEL spans - **Claude Agent SDK** — Built-in tracing with structured trace events ## Practical Tip: The Replay Debugger - Log every agent's input and output as structured JSON - When a workflow fails, replay individual agents with their exact inputs - This lets you test fixes on a single agent without rerunning the whole pipeline - Store traces for at least 30 days — patterns emerge over time ## Sources - [LangSmith Observability (LangChain)](https://docs.langchain.com/langsmith/observability) - [Braintrust: AI Observability Platform](https://www.braintrust.dev/) - [Arize Phoenix: Open-Source AI Observability](https://github.com/Arize-ai/phoenix) - [Claude Agent SDK (Anthropic)](https://github.com/anthropics/claude-agent-sdk-python) --- # Testing Multi-Agent Systems ## The Testing Pyramid for Agents ### Level 1: Unit Test Each Agent in Isolation - Mock the tools and feed the agent a known input - Assert the output matches expected structure and content - Test edge cases: empty input, malformed data, tool errors - Fast to run, cheap, catches most regressions ```python def test_researcher_agent(): mock_search = Mock(return_value=[{"title": "Paper A", "summary": "..."}]) result = researcher_agent.run( query="quantum computing trends", tools={"web_search": mock_search} ) assert "sources" in result assert len(result["sources"]) >= 3 ``` ### Level 2: Test Agent Pairs (Interface Tests) - Verify that Agent A's output format is valid input for Agent B - These catch schema mismatches between agents - Run the real Agent A, then validate its output against Agent B's expected input schema - Example: Researcher output should be a list of dicts with "title", "url", "summary" keys ### Level 3: Integration Test the Full Workflow - Run the complete pipeline end-to-end with real (or recorded) tool responses - Use deterministic inputs with known expected outputs - Assert on the final output quality, not intermediate steps - These are slow and expensive — run them nightly, not on every commit ### Level 4: Evaluation Suites - Build a dataset of 20-50 representative inputs with expected outputs - Score outputs using an LLM judge or human ratings - Track quality over time as you change agent prompts or models - Regression detection: alert if average quality drops below threshold ## Anti-Pattern: Only Testing the Happy Path - Test what happens when the web search returns zero results - Test what happens when the coding agent produces code that fails tests - Test what happens when an agent exceeds its iteration limit - Test what happens when two agents produce contradictory information --- # Key Takeaways ## Start Simple Don't reach for multi-agent when a single agent with good tools will do. Multi-agent adds latency, cost, and complexity. Earn your way into it by hitting a real single-agent ceiling first. ## Specialize Aggressively The power of multi-agent is specialization. Each agent should have a narrow role, a focused prompt, and only the tools it needs. If an agent's system prompt is longer than 200 words, it's probably doing too much. ## Design for Failure Every agent will fail eventually. Build structured error reporting, retry logic, and fallback agents into your system from day one. The orchestrator's most important job is handling failures, not just the happy path. ## Control Costs Proactively Set token budgets, iteration caps, and early stopping conditions before you ship. A collaborative loop without a max iteration count will drain your API budget overnight. Monitor cost per workflow, not just cost per call. ## Observe Everything Log every agent's input, output, tool calls, and timing. When a 4-agent pipeline produces a bad result, you need to replay each step to find the bug. Invest in tracing infrastructure early — it pays for itself on the first production incident. ## Test at Every Level Unit test agents in isolation with mocked tools. Integration test the full pipeline with known inputs. Build evaluation suites to track quality over time. Multi-agent bugs often live at the seams between agents, not inside any single one. --- # Key Research Papers Multi-agent AI systems are an active area of research. These papers provide the foundations for the patterns we covered today. | Paper | Authors | Year | Key Contribution | |---|---|---|---| | [**AutoGen**](https://arxiv.org/abs/2308.08155) | Wu et al. (Microsoft) | 2023 | Conversation-driven multi-agent framework; introduced the concept of "conversable agents" that collaborate through chat | | [**CAMEL**](https://arxiv.org/abs/2303.17760) | Li et al. | 2023 | Role-playing framework where agents take on personas and collaborate through structured dialogue; introduced inception prompting | | [**MetaGPT**](https://arxiv.org/abs/2308.00352) | Hong et al. | 2023 | Software engineering multi-agent system with SOPs; agents follow standardized operating procedures like a real dev team | | [**AgentVerse**](https://arxiv.org/abs/2308.10848) | Chen et al. | 2023 | Dynamic agent team composition; studies how to assemble the right group of agents for a given task | | [**ChatDev**](https://arxiv.org/abs/2307.07924) | Qian et al. | 2023 | End-to-end software development using a "virtual software company" of agents (CEO, CTO, programmer, tester, reviewer) | | [**Voyager**](https://arxiv.org/abs/2305.16291) | Wang et al. (NVIDIA) | 2023 | Lifelong learning agent in Minecraft; demonstrates skill library accumulation across agent iterations | | [**Self-Refine**](https://arxiv.org/abs/2303.17651) | Madaan et al. | 2023 | Iterative self-improvement without external feedback; foundational work for the reflection pattern | | [**Tree of Thoughts**](https://arxiv.org/abs/2305.10601) | Yao et al. | 2023 | Deliberate reasoning through exploring multiple solution paths; related to competitive/debate patterns | ## Recommended Reading Order 1. [**AutoGen**](https://arxiv.org/abs/2308.08155) — Foundational framework paper; covers most patterns 2. [**Self-Refine**](https://arxiv.org/abs/2303.17651) — Simple and powerful; the basis for reflection 3. [**MetaGPT**](https://arxiv.org/abs/2308.00352) — Best example of hierarchical multi-agent with structured roles 4. [**CAMEL**](https://arxiv.org/abs/2303.17760) — Understanding role-playing and collaborative dynamics ## Stay Current - Multi-agent research moves fast — new papers weekly on arXiv - Follow the `cs.AI` and `cs.MA` (multi-agent systems) categories - Framework documentation often outpaces published papers --- # Multi-Agent Workflows ## What We Covered - Six core patterns: Sequential, Hierarchical, Collaborative, Competitive, Supervisor, Swarm - Architecture fundamentals: agent design, communication, state management, tool isolation - Real-world examples: code review, research, customer support, data analysis - Production concerns: cost control, observability, testing ## The Guiding Principle **Use the simplest architecture that solves your problem.** Single agent > Multi-agent, until it isn't. --- *Next week: Human Oversight and Memory — approval gates, durable state, and retrieval.*