When to Use Multiple Agents (and When One Is Enough)

Multi-agent architectures have become the default answer to a question most teams have not asked yet. A single agent starts behaving inconsistently, someone suggests splitting it into a planner, a researcher, and a writer, and three weeks later the system is slower, costs four times as much, and fails in ways that are considerably harder to debug.

Sometimes multiple agents are exactly right. More often, the original problem was a context problem, a tool design problem, or a missing eval - and splitting the agent papered over it while adding a distributed systems problem on top.

This is how we decide, and how we build it when the answer is yes.

---

The Default Should Be One Agent

A single agent with well-designed tools and a clean state object handles more than people expect. Before splitting, we work through the failure honestly:

Is the system prompt trying to do four unrelated jobs? That is a context structuring problem. Conditionally include only the instructions relevant to the current step rather than shipping all four sets on every call.

Is it calling the wrong tools? That is usually tool design. Twenty overlapping tools with vague descriptions produce confused tool selection. Consolidating to eight well-named tools with sharp boundaries fixes more misbehavior than any orchestration change.

Is it losing the thread on long tasks? That is state management. Working state belongs in an explicit object the orchestration layer owns, not accumulated in conversation history.

Is quality inconsistent in ways you cannot characterize? You do not have an architecture problem yet. You have a measurement problem, and adding agents will make it worse by multiplying the places where quality can vary.

Splitting an agent you cannot measure gives you several agents you cannot measure.

---

When Multiple Agents Genuinely Earn Their Cost

Four situations where we reach for it deliberately.

Genuinely parallel work. When a task decomposes into independent subtasks whose results combine at the end - researching six suppliers, analyzing twelve documents, checking a claim against four sources - running them concurrently is a real latency win. The key word is independent. If subtask three needs the output of subtask two, this is a pipeline, not a fan-out, and one agent with a loop is simpler.

Context isolation as the goal. Sometimes you want a subtask to run without the main context polluting it, and without its intermediate work polluting the main context back. A sub-agent that reads forty pages and returns a three-paragraph summary keeps thirty-seven pages of noise out of the parent context. This is the most underrated reason and often the strongest one.

Different trust levels. When one step processes untrusted input - parsing a user-uploaded document, reading scraped web content - isolating it in an agent with a restricted tool set contains the blast radius of a prompt injection. The isolated agent cannot reach the tools that matter because it was never given them.

Adversarial review. Having a second agent critique the first output, with a different framing and no attachment to the original reasoning, catches a class of errors that self-review does not. This is the one case where redundancy rather than division of labor is the point.

Notice what is not on this list: "the prompt got long" and "it felt cleaner conceptually". Neither survives contact with the operational cost.

---

The Costs You Are Signing Up For

Be honest about these before committing.

Latency compounds on sequential paths. Each hop adds a full model round trip. A four-stage chain where each stage takes three seconds is a twelve-second response, and that is before retries.

Cost multiplies. Each agent carries its own context. Handing off means re-establishing enough context for the next agent to work, and that re-establishment is paid for in tokens on every handoff.

Errors compound quietly. Four stages at 95 percent reliability is 81 percent end to end. Worse, a subtly wrong output from stage one becomes confident input to stage two, and by stage four the error has been laundered into something that looks authoritative.

Debugging gets substantially harder. "Why did it do that?" now requires reconstructing a conversation between several non-deterministic components. Without tracing that spans the whole run, this is close to impossible.

---

Patterns That Work

When we do go multi-agent, these are the shapes we use.

• Supervisor with specialists. One agent owns the task and delegates to specialists with narrow tool sets. State lives with the supervisor. Specialists are effectively expensive, smart tools. This is our default because control flow stays in one place.

• Fan-out and synthesize. Split independent work across N parallel agents, collect results, synthesize. Best latency profile of any pattern, but only valid when the subtasks truly do not depend on each other.

• Pipeline with typed handoffs. Fixed stages, each with a schema-validated input and output. Works well for genuinely sequential transformation. The typed contract between stages is what keeps it debuggable.

• Generate and critique. One agent produces, another reviews against explicit criteria, the first revises. Cap the loop at two rounds. Beyond that we have never seen it converge on anything better.

What we avoid: free-form agent conversation where several agents talk until they agree. It is unbounded in cost, unpredictable in output, and nearly impossible to eval. Every time we have seen it in a production system, it was replaced within a quarter.

---

Handoffs Are the Hard Part

Most multi-agent failures are handoff failures, not reasoning failures.

Handoffs must be typed. Agent A passes a validated object to agent B, not a paragraph of prose that agent B has to reinterpret. Prose handoffs are where information silently goes missing. If the contract is a schema, a malformed handoff fails at the boundary instead of three stages downstream.

Pass state, not transcripts. The receiving agent needs the facts, not the reasoning that produced them. Passing full conversation history between agents is how you get four agents each carrying the accumulated context of everyone before them.

Decide who owns the failure. When a specialist fails, does it retry, escalate to the supervisor, or fail the whole task? Pick per specialist, up front, and enforce it in code rather than hoping the supervisor infers the right behavior.

Make the whole run one trace. Every agent call, tool call, and handoff should share a trace ID so you can reconstruct the entire run afterward. Without this, the system is effectively unobservable, and unobservable systems do not get fixed - they get rewritten.

---

Evaluating a Multi-Agent System

End-to-end evals are necessary and not sufficient. When the whole run fails, they tell you nothing about which stage caused it.

We eval each agent independently against its own contract - given this input, did it produce a valid, correct output - and separately eval the full pipeline. Stage-level evals localize regressions; end-to-end evals catch the compounding failures that only appear when the stages interact.

Worth tracking specifically: handoff validity rate, per-stage success rate, and how often the supervisor escalates versus retries. Those three numbers usually explain most of what is going wrong.

---

Cost and Latency Control in a Multi-Agent System

Once you have committed to multiple agents, these are the levers that keep the system affordable.

Do not use the strongest model everywhere. A supervisor doing routing and synthesis needs strong reasoning. A specialist extracting structured fields from a document usually does not. Assigning models per role rather than per system is often a large cost reduction with no measurable quality change - and the per-stage evals are what let you prove that rather than guess.

Cap the total step count. Every multi-agent system needs a hard ceiling on total model calls per task, enforced in the orchestration layer. Without it, a supervisor and a specialist can hand work back and forth in a loop that only ends when someone notices the bill. We set the cap at roughly twice the expected worst case and alert when a run approaches it, because runs that approach the ceiling are usually a bug.

Parallelize anything that can be parallelized. In a fan-out, running six subtasks concurrently rather than sequentially is the entire reason to use the pattern. It is worth being deliberate that the implementation actually does this - it is easy to write what looks like a fan-out and have it execute in series.

Cache at the specialist boundary. Specialists with narrow, well-defined inputs are far more cacheable than a general agent. If the same document is analyzed twice, the second call should be free.

Watch for context re-establishment cost. Each handoff re-sends whatever the next agent needs to know. In a chatty system this can exceed the cost of the actual reasoning. If the handoff payload is growing, that is a signal the split is in the wrong place.

---

Failure Handling Across Agents

Single-agent failure handling is mostly retries and guardrails. With multiple agents there is a new question at every boundary: when one component fails, what should the rest of the system do?

Distinguish transient from terminal failures. A timeout calling an API is worth retrying. A specialist returning output that fails schema validation twice in a row is not - it will fail a third time. Retrying terminal failures is how a system that should have failed in four seconds fails in ninety instead.

Decide what a partial result is worth. In a fan-out over six sources where two fail, is a synthesis of four acceptable? Sometimes yes, with a clear caveat in the output. Sometimes the whole task is invalid without complete data. This is a product decision that has to be made explicitly, because the default behavior of most implementations is to silently synthesize whatever came back.

Never let a failure become invisible input. The worst pattern is a specialist that fails and returns an empty or apologetic string, which the supervisor then treats as a legitimate finding. Failures must be typed as failures so the supervisor can branch on them, not passed along as prose that reads like a result.

Give the whole task a deadline. Individual timeouts do not bound total latency when stages are sequential and each retries. A wall-clock budget for the entire run, enforced by the orchestrator, is what actually protects the user experience.

---

Where to Start

Start with one agent and instrument it well enough to know exactly how it is failing. Fix the tool design and the context structure first, because those fixes are cheap and they frequently dissolve the problem entirely.

If a genuine case remains - real parallelism, real context isolation, a real trust boundary - split along that specific seam and nothing else. One well-justified split beats a five-agent architecture built on an intuition about how the work ought to be divided.

The foundations this all sits on - typed tools, explicit state, guardrails in the tool layer - are covered in Architecting Production LLM Agents. If you are weighing this decision on a system that is already live, our AI Agent Development practice does this kind of assessment.

Add an agent when the architecture demands it, not when the prompt gets uncomfortable.