Multi-Agent Communication

A message is a block of tokens, read twice

Start with the unit. When agent A sends agent B a message, that message costs tokens at least twice: once when A generates it (decoding, one token at a time) and again when B reads it (prefill, ingesting it into B’s context). If the message is broadcast to several agents, the read cost is paid once per recipient. So the natural cost model for a message of m tokens sent to r recipients is:

cost(message) = m (generate)  +  r × m (each recipient prefills)

The generation term is fixed by whoever speaks; the r × m read term is where topology enters, because r is decided entirely by who is wired to hear whom. And crucially, prefill is not a one-time charge: on a stateless chat API every agent re-sends its whole context every turn, so a message that lands in a shared transcript is re-read on every subsequent round, not just the round it arrived. That re-reading, compounded over rounds, is what turns a modest conversation into a runaway bill.

Shared state versus private state

The first design fork is whether agents share a single context or each keep their own. A shared or ‘blackboard’ design gives every agent the full transcript: everyone sees everything, so coordination is easy and nobody misses a fact. A private design gives each agent only the slice it needs — its own task and whatever was explicitly forwarded to it.

The tradeoff is pure token economics. Shared state means every message enters a transcript that every agent re-ingests each turn, so redundant reading grows with both the number of agents and the number of messages. Private state removes that redundancy — an agent never pays to read a conversation it is not part of — but it risks divergence: two agents can now hold contradictory pictures of the world and never notice. Put bluntly, shared state buys coherence with tokens; private state saves tokens and risks incoherence. Most robust systems land in between: private working context plus a small, deliberately-shared summary of the facts everyone must agree on.

Topology decides the fan-out

The communication graph — who can talk to whom — is the single biggest lever on cost, because it fixes the recipient count r for every message. Three shapes cover most real systems:

Edge count is the headline. A star and a chain both connect N agents with only N − 1 links, so communication scales linearly. A full mesh has O(N^2) links, and if every agent broadcasts to every other each round, the token traffic scales like N^2 too. The jump from linear to quadratic is not a rounding error — it is the difference between a team that scales to dozens of agents and one that chokes at five.

The quadratic cost of a full mesh

Make the mesh cost concrete. Suppose N agents each speak once per round with a message of m tokens, and every message is broadcast to the other N − 1 agents. The read (prefill) traffic for a single round is:

reads_per_round = N × (N − 1) × m  ≈  N^2 · m

Now add the compounding. On a shared transcript, round t forces each of the N agents to re-read everything said in rounds 1…t−1. Summing the growing transcript over T rounds gives:

total_reads ≈ N^2 · m · Σ_{t=1}^{T} (t − 1)
            = N^2 · m · T(T − 1)/2   =   O(N^2 · m · T^2)

Quadratic in agents and quadratic in rounds. This double-quadratic is the signature failure mode of naive ‘let all the agents debate’ designs: doubling the team quadruples the fan-out, and letting the debate run twice as long quadruples the re-reading. It looks fine in a three-agent demo and detonates in production.

A worked token budget

Plug in numbers. Take N = 5 agents, messages of m = 300 tokens, running for T = 6 rounds. In a full-mesh shared transcript, the cumulative read cost is:

total_reads = N^2 · m · T(T−1)/2
            = 25 × 300 × (6·5/2)
            = 25 × 300 × 15  =  112,500 tokens

Generation is almost a rounding error next to that: N · m · T = 5 × 300 × 6 = 9,000 tokens. So roughly 92% of the compute goes to agents re-reading each other, not to producing anything new. Now switch to a star where the orchestrator compresses each round into a d = 150-token digest and workers see only that digest plus their ~200-token task. Worker reads become N · T · (d + task) = 5 × 6 × 350 = 10,500; the hub reads N · m replies per round, 5 × 300 × 6 = 9,000. Total ≈ 19,500 tokens — the same six rounds of collaboration for under a fifth of the cost.

Bandwidth versus consensus

Underneath topology sits a genuine tradeoff you cannot design away: bandwidth versus consensus. More communication — more messages, wider broadcast, more rounds — gives agents a more complete shared picture and helps them converge on a consistent answer. Less communication is cheaper but risks agents working from stale or partial information, duplicating effort, or never agreeing at all.

The catch is that the returns on bandwidth diminish fast while the cost keeps climbing. The first exchange of information usually resolves most of the disagreement; the fifth round of a debate rarely changes the conclusion but costs as much as the first — more, actually, because the transcript is longer. Worse, unbounded discussion can oscillate, with agents politely flip-flopping without converging. The engineering answer is to spend bandwidth where consensus actually matters (the shared facts and the final decision) and starve it everywhere else (private scratch work), and to cap the number of rounds so a non-converging debate fails fast instead of burning the whole budget.

Protocol design: structure versus prose

The last lever is the form of the messages. Natural-language messages are flexible and expressive, but they are token-heavy and ambiguous — a receiving agent may parse ‘handle the edge case’ three different ways. Structured messages — typed JSON, a fixed schema of fields, an explicit ‘speech act’ like REQUEST / INFORM / PROPOSE — are more compact per unit of meaning and far less ambiguous, at the cost of expressiveness and some rigidity.

A useful rule: the more agents and rounds you have, the more structure pays off, because ambiguity compounds just like tokens do. A two-agent draft-and-critique loop can afford to chat in prose; a ten-agent system negotiating a plan wants a tight schema so that parsing is deterministic and the token cost per message is predictable. Structure also makes the channel auditable — you can log, validate, and route typed messages, which matters once the system is big enough that you can no longer read every exchange by hand. Most mature protocols keep a small structured envelope around an optional free-text payload, getting both.

What this means for CPU-bound small models

All of the above gets sharper when the models are small language models running on a CPU. Two facts dominate. First, the context window is small — a growing shared transcript hits the ceiling quickly, so the T^2 blow-up is not just expensive, it is fatal, truncating exactly the history the agents need. Second, prefill on a CPU is slow: without a GPU’s parallelism, ingesting a long context is a real wall-clock cost, so every redundant re-read is felt as latency, not just token count.

The design consequences are clear. Prefer star or chain topologies over full meshes; keep shared state small and summarized rather than a growing transcript; use structured, compact messages to squeeze meaning into few tokens; forward deltas instead of whole histories; and cap the number of rounds hard. A CPU-SLM agent team that respects the linear-not-quadratic rule can coordinate usefully; one that lets five small models free-associate in a shared mesh will spend all its time re-reading and none of it thinking.