MCP streaming transport architecture — SSE, chunked, backpressure

Why architecture matters here

Streaming fails on proxy buffering + missing timeouts. Architecture matters because SSE + chunks + backpressure compose.

The architecture: every piece explained

The top strip is basics. Client. HTTP request. SSE stream. Chunked frames.

The middle row is dataflow. Progress events. Partial results. Backpressure. Cancel.

The lower rows are ops. Reconnect. Metrics. Ops — proxy + timeout + observability.

End-to-end flow

End-to-end: client POSTs tool call with Accept: text/event-stream. Server holds connection open, sends notifications/progress events with delta text. Client renders incrementally. On done: final result event + stream close.

Streamable HTTP: one endpoint instead of two

The wire sketched above is the transport MCP standardises as Streamable HTTP, introduced in the 2025-03-26 revision to replace an earlier two-endpoint design. Its entire surface is a single URL - the MCP endpoint - that accepts POST and, optionally, GET.

The streaming rationale for that collapse is the important part. In the older shape the event stream was the only path from server to client, so it had to exist before any work was requested and stay up for the life of the session. A four-millisecond tools/list returning 900 bytes paid for a long-lived connection it had no use for, and the server had no way to simply answer with a JSON body. Streamable HTTP inverts that: streaming becomes a per-response decision, made when the server decides how to answer.

So it is a spectrum, not a mode. A server that streams only its slow tools pays streaming's operational cost - held sockets, buffering-hostile proxies, idle timeouts - on the fraction of traffic that benefits, and behaves like an ordinary JSON-RPC-over-HTTP service otherwise. The abstraction underneath is MCP transport architecture's subject; what follows is the HTTP mechanism.

One endpoint, two methods

POST carries client-to-server traffic. The client sends a JSON-RPC message in the body and must advertise both possible answers with Accept: application/json, text/event-stream - offer only one and there is no legal response the server can give. Arrays of messages were permitted when Streamable HTTP landed and JSON-RPC batching was removed again in the 2025-06-18 revision, so batch support is revision-dependent rather than assumable.

What comes back depends on what went out. A body holding only responses and notifications - messages needing no answer - gets 202 Accepted with an empty body. A body holding a request gets either a single JSON object with Content-Type: application/json or an SSE stream with Content-Type: text/event-stream. The choice is the server's, per request, and the client must implement both branches. Clients that switch on the request method rather than the response content type break the first time a normally-fast tool decides to stream.

GET opens a stream for server-initiated messages belonging to no in-flight request: notifications/resources/updated, notifications/tools/list_changed, log records delivered as notifications/message. A server with nothing to push may answer 405 Method Not Allowed, which a client must read as "this server never initiates" rather than as something to retry. One rule binds the two methods: a given message is delivered on exactly one stream, so a server must never fan the same notification across every stream a client holds open.

When a POST answers with a stream

A streaming answer is an ordinary HTTP/1.1 response with Transfer-Encoding: chunked and an event-stream content type, which is why the whole path must speak HTTP/1.1 or better - a hop that downgrades to 1.0 cannot express chunked framing and will buffer or truncate.

Three kinds of traffic ride that stream. Progress notifications and log messages flow toward the client. So do server-to-client requests - sampling, elicitation, roots - and those catch implementers out, because the client's reply does not travel back up the stream; it goes out as a new POST to the same endpoint while the original stream stays open. Third and terminally, the JSON-RPC response for each request in the originating POST, after which the server should close the stream.

Since closure is a signal and not a guarantee, a client must decide completion by having seen a response object for every request ID it sent. A stream that ends with a request outstanding is a failure, and at the socket layer it is indistinguishable from a proxy timing out - which is why resumption exists.

Note what the transport does not give you: a token-delta primitive. A tool result is one JSON-RPC response, so "streaming a tool's output" means a run of progress notifications ahead of one terminal result. Aborting early is likewise protocol-level, in MCP cancellation - dropping the TCP connection is not a cancellation, and a server that treats it as one orphans work on every proxy hiccup.

Session identity on the wire: the Mcp-Session-Id header

HTTP has no connection to hang a session on, so Streamable HTTP puts the session in a header. On the response carrying the InitializeResult, a server may assign Mcp-Session-Id. The value must be globally unique and cryptographically secure - a UUIDv4 or a signed token, never an incrementing integer - and is restricted to visible ASCII, roughly 0x21 through 0x7E, so it can never carry a space or newline that would break header framing. Once assigned, the client must echo it on every subsequent request, POST and GET alike, and a server may reject one that omits it with 400 Bad Request. A server that assigns no ID is declaring itself sessionless; the client must then send no such header rather than inventing one.

Two status codes carry the rest of the lifecycle. A server may terminate a session whenever it likes - eviction, deploy, idle timeout - and afterwards answers 404 to any request bearing that ID. The correct client behaviour on 404 is to start over: a fresh initialize with no session header, then replay whatever state it needs. Treating 404 as fatal is the most common bug in home-grown clients, and it turns a routine restart into a dead agent. In the other direction the client should send an HTTP DELETE carrying the header to end a session explicitly; a 405 there means the server reserves termination for itself.

A companion header arrived in the 2025-06-18 revision: MCP-Protocol-Version, sent on every request after initialization so a server fronting several revisions can dispatch without re-reading the handshake. The session ID also doubles as the load-balancer affinity key when server state lives in process - route by it or externalise the state, a tradeoff that is MCP sessions' subject.

Resumability: event IDs and Last-Event-ID

SSE has a built-in resumption mechanism and Streamable HTTP adopts it. A server may attach an id: field to each event it emits. If it does, those IDs must be globally unique within the session - or across all clients, if the server is sessionless - because they are the only cursor either side has.

When a stream breaks, the client reconnects with Last-Event-ID set to the last ID it actually processed. The server may then replay the messages that would have followed that event on that stream and continue. The scoping is strict and load-bearing: replaying another stream's messages would duplicate a response the client already handled and reorder notifications relative to their originating request.

That scoping is what makes the server side non-trivial. Replay needs a per-stream ring buffer of recent events, and its depth is literally your maximum tolerable disconnect - a few dozen events, sized from an observed notification rate rather than guessed. Unbounded buffers turn a client that never returns into a slow leak, so evict on session teardown and on a hard age bound, and answer a stale Last-Event-ID by declining to resume rather than silently restarting from the beginning.

Two limits are worth stating plainly. Replay is at-least-once, so notification handling must be idempotent. And whether a POST-initiated stream can be resumed at all, and by which request, is loose in the spec text and diverges across implementations - treat it as revision- and implementation-dependent. The safe client assumes a lost POST stream may not be resumable and can re-issue the request, which means any non-idempotent tool call needs an idempotency key of its own.

A minimal exchange on the wire

POST /mcp HTTP/1.1
Host: tools.example.com
Content-Type: application/json
Accept: application/json, text/event-stream
Mcp-Session-Id: 1868a90c-3f2b-4d5e-9a71-0c4b8e2f6d11
MCP-Protocol-Version: 2025-06-18
Authorization: Bearer eyJhbGciOi...

{"jsonrpc":"2.0","id":7,"method":"tools/call","params":{"name":"reindex",
 "arguments":{"corpus":"docs"},"_meta":{"progressToken":"7-a"}}}

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache, no-transform
X-Accel-Buffering: no
Transfer-Encoding: chunked

id: 42
data: {"jsonrpc":"2.0","method":"notifications/progress","params":{"progressToken":"7-a","progress":128,"total":4096}}

: keepalive

id: 43
data: {"jsonrpc":"2.0","id":7,"result":{"content":[{"type":"text","text":"reindexed 4096 docs"}]}}

Four details do the work. The status is 200, not 202 - the 202 path is reserved for bodies that asked nothing. The id: lines are the resumption cursor and cost nothing until a connection drops. The bare : keepalive line is an SSE comment: every conforming parser discards it, it never reaches the JSON-RPC layer, and it is the only portable way to keep an idle stream warm. And the last frame is the response to request 7, which is what ends the exchange - the socket closing afterwards is a consequence, not the signal.

Proxies, buffering, timeouts, and backpressure

Streaming code that works against a local server and dies behind the load balancer is the signature failure of this transport, and the cause is almost never the application.

The path between you and the client

nginx buffers proxied responses by default. With proxy_buffering on it accumulates events and forwards them when a buffer fills or the upstream closes - converting a stream into one lump at the end, with every symptom of a hang and none of an error. Turn it off for the location, or emit X-Accel-Buffering: no on streaming responses only; nginx honours that per response, so one location can serve both buffered JSON and unbuffered streams. The proxy module also defaults to proxy_http_version 1.0, which cannot express chunked transfer coding, and proxy_read_timeout defaults to 60 seconds measured between reads, so a stream that thinks quietly for a minute is killed mid-flight. Compression middleware buffers in order to compress; exclude text/event-stream.

location /mcp {
    proxy_pass          http://mcp_upstream;
    proxy_http_version  1.1;      # 1.0 default cannot do chunked
    proxy_set_header    Connection "";
    proxy_buffering     off;      # or emit X-Accel-Buffering: no
    proxy_cache         off;
    proxy_read_timeout  3600s;    # default 60s kills idle streams
    proxy_send_timeout  3600s;
    gzip                off;
}

Managed infrastructure has the same edges with less visible configuration. An AWS Application Load Balancer idles out at 60 seconds by default; API Gateway buffers whole responses and so cannot carry SSE at all, while a Lambda Function URL with response streaming can. The fix that survives all of them is a keepalive comment every 15 to 30 seconds, comfortably under the tightest timeout on the path. It doubles as liveness detection, since writing to a peer that has vanished is how the server finds out.

Backpressure without acknowledgements

SSE carries no application-level acknowledgement, so there is no MCP-layer credit or window to consult. Flow control is entirely the TCP receive window and, over HTTP/2, per-stream credit. A slow consumer surfaces on the server as a write that blocks, returns EWOULDBLOCK, or leaves an async framework awaiting drain - and if that signal is ignored, pending bytes pile up in the process rather than on the network.

The lever is a bounded per-stream queue with an explicit overflow policy. Progress notifications are the compressible traffic: keep only the newest per progress token, since a counter's intermediate values carry nothing. Responses and requests are never droppable, and if the queue overflows on those, close the stream and let resumption recover it rather than growing until the process is OOM-killed and every session on it dies at once. Watch queue depth and per-stream write-stall time; both go bad long before any error rate does.

Guarding a publicly reachable endpoint

Making the transport an ordinary HTTP endpoint makes it reachable by anything that speaks HTTP, including a web page the user happens to have open. The spec's hard requirement is that servers validate the Origin header on all incoming connections, and the attack it blocks is DNS rebinding: a local server on 127.0.0.1:3000 becomes addressable from any site once an attacker's hostname resolves to loopback, and without an origin allowlist that site drives whatever filesystem or shell tools the server exposes. Bind local servers to 127.0.0.1 rather than 0.0.0.0 - a one-word change that removes the whole LAN from the threat model.

Credential placement then differs between the two methods, and that is genuinely transport-shaped. A POST is a normal request whose token is evaluated on arrival. The GET stream is one request that may live for an hour, so its authorization is checked once, at open: a token expiring twenty minutes in is never re-examined unless the server re-examines it. Bound stream lifetime to token lifetime, or revalidate on a timer and close on revocation. There is no way to send a 401 down a response that already returned 200 - the best a server can do is end the stream and reject the next POST.

Finally, the session ID is a routing and state key, not a credential, and must never stand in for authentication however unguessable it looks. Token acquisition, dynamic registration and audience binding are MCP authorization's subject.