Why architecture matters here
Bidi streams promise a nice model — write messages in either direction — but production surprises come from HTTP/2 internals. A slow reader consumes flow-control window; the sender stalls silently. A load balancer forces GOAWAY; streams drop mid-conversation. Keepalive misconfigured means half-open connections linger for hours.
The architecture matters because the fix is not "use gRPC"; it is understanding how HTTP/2 works and configuring the client, server, and load balancer to cooperate.
With the pieces mapped, a bidi service becomes a durable long-lived resource, not a fragile session.
Two independent halves, not a conversation
The general shape of gRPC — the HTTP/2 framing, the five-byte length prefix, the four call types, protobuf, deadlines, load balancing, retry policy — is covered in @@NET@@. This article assumes all of it and looks only at the one call type that behaves differently from everything else: bidirectional streaming.
A bidi RPC is a single HTTP/2 stream carrying two logically independent message sequences. The client's half is the sequence of DATA frames it writes; the server's half is the sequence it writes back on the same stream ID. They share a stream, a connection and a deadline, and nothing else. There is no protocol-level notion that message 7 from the server is a reply to message 4 from the client. The framing does not carry a correlation field, the runtime does not track one, and the two halves are not even required to be active at the same time — a client may write for an hour before the server writes anything, or the reverse.
Almost every bidi bug starts as a request/response assumption smuggled into a full-duplex channel. If your protocol genuinely needs pairing — a request that has an answer, a cancel that has an ack — then you carry a request_id in the message and match on it yourself. If you skip that, the first time the server emits an unsolicited event (a push notification, a server-initiated heartbeat, a mid-stream error hint) your client will attribute it to whatever it happened to send last.
Half-close: CloseSend is not cancel
Ending the write half is called half-close. In Go it is stream.CloseSend(), in Java requestObserver.onCompleted(), in Python await call.done_writing(). Whatever the spelling, the wire effect is the same: the END_STREAM flag is set on the client's last DATA frame (or on an empty one, if there is nothing left to send). It means exactly "I will send no more messages". It does not end the RPC.
The server may keep writing indefinitely afterwards. The RPC ends only when the server sends its trailing HEADERS frame carrying grpc-status. Two rules fall out of that, and both are routinely broken:
You must keep reading after you half-close. The status lives in the trailers, so a client that half-closes and then walks away never learns whether the call succeeded. Drain reads until you get EOF (Go), onCompleted (Java) or StopAsyncIteration (Python) — that is the moment the status is known, and it is the only place a clean success is reported.
There is no server-side half-close. HTTP/2 has no way for the response half to end while the request half stays open. A server that stops writing has ended the RPC, full stop. So a "server is finished, client keeps uploading" phase has to be modelled as an application-level message, never as a transport event.
Finally, abandoning the stream object without half-closing is not a polite exit: the binding sends RST_STREAM and the server's handler observes CANCELLED. Cancel and half-close look similar in application code and are opposites on the wire.
The architecture: every piece explained
The top strip is the framing. Client stream sends messages via HTTP/2 frames (HEADERS, DATA). Multiplexed streams share one TCP connection; each stream has its own ID and flow state. Server stream writes back on the same connection.
The middle row is the control mechanisms. Flow control uses window updates at both connection and stream level; when the receiver's window is zero, the sender must stop until a WINDOW_UPDATE arrives. Backpressure is how the application observes this — the async sender pauses on write when the wire is full. Deadlines are per-call timeouts; deadlines propagate downstream through child calls. Keepalive + ping detect dead connections; misconfigured they cause noise, correctly configured they save you.
The lower rows are resilience. Reconnect + retries use exponential backoff with jitter; hedging and retry policies must be written for idempotent semantics. Load balancer is critical — L4 balancers pin to one backend; L7 (or client-side name resolution + subchannels) enables real load balancing across backends. Observability tracks per-stream latency, flow control stalls, and GOAWAY events.
Driving both halves concurrently, or deadlocking
Here is the failure that costs the most production time. A developer writes the obvious loop — send a message, read the reply, repeat — on one thread. It passes every test, because in tests the messages are small and both flow-control windows have headroom, so Send returns immediately.
In production the peer is also writing. Your process is not reading, so its receive window drains to zero and it stops sending WINDOW_UPDATE. The peer's write blocks. Because the peer is blocked in its write, it never reaches its own read, so its receive window drains too. Now your Send blocks. Both processes are parked in a write, neither is reading, and nothing will ever unstick it. There is no error, no log line, no CPU burn — just two healthy-looking processes and a stream that stopped. Absent a deadline or a keepalive timeout the connection sits like that until someone restarts it.
The rule is unconditional: the read half and the write half must be driven by separate schedulable units. Never let one of them be a prerequisite for the other making progress. The bindings differ in how you say it:
Go — a goroutine per direction over the same ClientStream. The concurrency contract is that one goroutine may call SendMsg and another may call RecvMsg concurrently, but two goroutines may not call SendMsg at the same time; serialise your writers behind a channel.
Java — the async stub gives you a StreamObserver per direction, but the naive version writes from inside onNext, which is the transport callback thread; blocking there blocks the whole channel. Cast the request observer to ClientCallStreamObserver, register setOnReadyHandler, and only write while isReady() is true. On the receive side call disableAutoRequestWithInitial(1) and then request(n) per consumed message, so messages are pulled instead of pushed.
Python asyncio — two tasks: one async for over the call, one awaiting call.write(). The synchronous API's blocking iterator on the same thread as the writer is the canonical deadlock.
C++ — the completion-queue rule is that at most one Read and one Write may be outstanding per stream at a time. Issue the next Read from the completion tag of the previous one; queue outbound messages and start the next Write only when the last one completes.
stream, err := client.Chat(ctx)
if err != nil { return err }
// write half - its own goroutine, nothing else may call Send
go func() {
defer stream.CloseSend() // END_STREAM on our last DATA frame
for msg := range outbound {
if err := stream.Send(msg); err != nil {
return // the real status comes from Recv, not here
}
}
}()
// read half - drain to EOF so the trailers are actually consumed
for {
in, err := stream.Recv()
if err == io.EOF { // clean close: grpc-status OK in trailers
return nil
}
if err != nil { // non-OK status, or RST_STREAM
return err
}
handle(in) // must not block forever: see below
}Note the comment on the last line. If handle blocks, you have simply moved the stall from the transport into your loop — which, as the next section explains, is sometimes exactly what you want.
Flow control: two windows, and the one everyone forgets
HTTP/2 flow control is credit-based and exists at two levels: one window per stream and one window per connection, each starting at 65,535 bytes and each replenished by its own WINDOW_UPDATE frames. Both must have credit for a DATA frame to go out. The per-connection window is the one that gets forgotten, and it is why a single misbehaving bidi stream degrades every other RPC sharing that channel: the stalled stream holds unacknowledged bytes against the connection window, and healthy streams whose own windows are wide open still cannot write.
The 64 KiB default is small enough to be the binding constraint on any fat pipe — at 100 ms round trip it caps a single stream near 640 KB/s regardless of link capacity. Mature implementations therefore auto-tune from a bandwidth-delay estimate rather than leaving the default in place, and you can also set it explicitly: grpc.WithInitialWindowSize and grpc.WithInitialConnWindowSize in Go (setting either one disables that dynamic tuning, so measure before you pin it), NettyChannelBuilder.flowControlWindow in Java.
conn, err := grpc.NewClient(target,
grpc.WithInitialWindowSize(1<<20), // per-stream: 1 MiB
grpc.WithInitialConnWindowSize(4<<20), // per-connection: 4 MiB
grpc.WithKeepaliveParams(keepalive.ClientParameters{
Time: 30 * time.Second,
Timeout: 10 * time.Second,
PermitWithoutStream: false, // do not ping an idle connection
}),
)How the window gets defeated
The window only produces backpressure if your application actually declines to consume. The default behaviour of several bindings is to drain the socket eagerly into an unbounded in-process queue and hand you messages from there. The receive window then never closes, WINDOW_UPDATE goes out the instant bytes arrive, the sender sees a permanently open pipe, and the first symptom of a slow consumer is heap exhaustion rather than a slowed producer. Backpressure did not fail — it was never connected.
Reconnecting it is binding-specific. In Java it is disableAutoRequest plus explicit request(n). In Go the coupling is there by default, because RecvMsg pulls from the transport buffer: not calling RecvMsg is the backpressure — which you throw away the moment you spawn a goroutine per received message. In Python asyncio, awaiting per message inside the async for keeps the coupling; pushing onto an unbounded asyncio.Queue breaks it.
That is the whole gRPC-specific mechanism. The general theory — credit accounting from source to sink, overflow policies, drop versus block versus shed — is covered in @@SBP@@, @@BBP@@, and @@ABP@@ for the agent-boundary case.
Ordering within a direction, and none across
Within one direction of one stream, delivery is strict FIFO and lossless for as long as the stream is alive: it is a single TCP byte stream, so messages cannot overtake each other and cannot be silently dropped — if something goes wrong the stream errors rather than skipping a message.
Across directions there is no ordering relationship whatsoever, and this is the guarantee people assume without checking. The server processing message 5 while you receive its response to message 2 is completely normal. So is receiving an unsolicited server message between two of your own writes. HTTP/2 also interleaves DATA frames from all streams on the connection according to the sender's scheduler, so relative timing between two different streams is not something you can reason about at all — if two messages must be ordered with respect to each other, they must travel on the same stream in the same direction.
None of the above survives a reconnect: a new stream is a new ordering domain. Sequence numbers, reorder windows and per-key ordering across a fanout are a distributed-systems problem rather than a transport one, and are covered in @@ORD@@.
End-to-end flow
End-to-end: a chat client opens a bidi stream to the server. Client sends the first message; HTTP/2 frames flow. Server responds. Both stream for hours. Server needs to drain for a deploy; it sends GOAWAY with last-stream-id. Existing streams continue until natural close; new streams route to a fresh backend. Meanwhile, a slow client stops reading; flow control window drops to zero; server's write blocks; backpressure propagates. Keepalive pings run every 60s; a dead peer is detected within 2 pings and connection closed. Observability shows median stream age, GOAWAY count, and reconnect success rate. Load balancer is L7 gRPC-aware; new streams distribute properly.
Keepalives and connection age on hour-long streams
An idle bidi stream is indistinguishable from a dead one. Nothing is in flight, so NAT tables, cloud load-balancer idle timers and stateful firewalls are free to drop the flow, and neither end finds out until the next write hits a TCP retransmission timeout minutes later. HTTP/2 PING keepalives are the answer, with the client's ping interval set comfortably below the shortest idle timeout in the path. Server-side ping enforcement, the too_many_pings GOAWAY and the reconnect storm it causes are described in @@NET@@; the setting that matters for bidi specifically is PermitWithoutStream, which should stay false when your streams are long-lived, because there is always an active stream to justify the ping.
The genuinely bidi-specific problem is maximum connection age. Recycling connections with MaxConnectionAge plus MaxConnectionAgeGrace is the standard way to rebalance a fleet after a scale-out: the server sends GOAWAY with a last-stream-ID, in-flight calls finish inside the grace period, the connection closes, clients re-resolve. That reasoning silently assumes calls are short. A bidi stream that lives for four hours will not finish inside a thirty-second grace window, so age-based recycling degenerates into scheduled mid-conversation terminations.
There are only three honest options. Exempt streaming backends from age-based recycling and rebalance some other way. Set the grace period long enough that your streams' 99th-percentile lifetime fits inside it, which for chat or telemetry usually means it is not a real bound. Or send an application-level drain message — a "please reconnect now" control frame — and let the client close and re-dial at a moment of its own choosing, with jitter so a whole fleet does not stampede. The third option is the only one that scales, and it exists because the transport has no graceful mid-stream migration primitive. General heartbeat and liveness design is in @@HB@@.
Cancellation and errors mid-stream
Every abnormal end of a bidi stream is one of two wire events: a RST_STREAM frame, or trailing HEADERS with a non-OK grpc-status. A client that cancels its context sends RST_STREAM; the server's handler context is cancelled and, if the handler propagates it, so is every downstream call it started. A server handler that returns an error sends trailers instead, and the client's read half surfaces that status.
The trap is which call reports it. When a stream breaks, the pending Send on the write half usually fails too — but with a generic transport error such as "the client connection is closing", not the real status. The actual grpc-status is delivered to the read half. So a client that logs the send failure, tears down its goroutines and never drains reads will report a useless error for what was in fact a clean RESOURCE_EXHAUSTED or FAILED_PRECONDITION. Always take the terminal status from the receive side, and treat write errors as "something ended, go look at the read half".
Deadlines behave differently here too. grpc-timeout bounds the entire stream, not a single message exchange, so a 30-second deadline on a chat stream kills the chat after 30 seconds. Long-lived streams either run with no deadline — relying on keepalives to detect death — or with a deliberately long one that functions as a maximum session length. Per-message timeouts, if you need them, are application state: record when you sent the request, and time it out yourself.
Reconnection: there is no replay log
gRPC's built-in retry policy does not rescue a bidi stream. An RPC is committed once the first response message reaches the client, and after commitment retry is off the table — so any stream that has produced output is unretryable by construction (see @@NET@@). Reconnection is therefore application logic, always.
What makes it hard is not re-dialling; it is that a broken stream tells you nothing about how much the peer received. Bytes you handed to Send may be sitting in a socket buffer, may have been received but not processed, or may have been fully applied. The transport will not distinguish those cases for you, and there is no replay: unlike a broker, a gRPC stream has no persistence, no consumer offset and no redelivery.
A resumable bidi protocol therefore carries its own bookkeeping. Number every message per direction. Acknowledge periodically — cumulative acks are cheap and enough. Retain unacknowledged messages in a bounded sender-side buffer, with an explicit policy for what happens when it fills. On reconnect, send a resume request carrying the last sequence number you successfully applied, and have the server either replay from there or reject the resume and force a full resync. Make the apply path idempotent so a replayed message is harmless, because you will replay. And reconnect with exponential backoff plus jitter and a ceiling: a thousand clients whose streams all died with one backend will otherwise re-dial in lockstep and take out the next one.
Proxies, idle timeouts, and head-of-line blocking
The middleboxes between client and server were mostly designed for short requests, and long-lived bidi streams find every place that assumption is baked in.
Response buffering. Any proxy that accumulates a response before forwarding it converts a streaming channel into a batch one, and the symptom is messages arriving in clumps with no error anywhere. nginx's grpc_pass streams rather than buffering, but a generic HTTP proxy in front of it, or a WAF doing body inspection, will not.
Idle timeouts. nginx applies grpc_read_timeout and grpc_send_timeout between successive reads and writes, with defaults on the order of a minute — a stream quieter than that is closed even though it is perfectly healthy. Envoy's route timeout bounds the whole stream and must be set to 0s for streaming routes, with idle_timeout used instead if you want a liveness bound. AWS ALB and NLB idle timeouts count a connection with no bytes in flight as idle regardless of how many streams are open on it. In every case the fix is the same shape: keepalive or application pings at an interval below the tightest timeout in the path, and that interval must be derived from the actual proxy config rather than guessed.
L4 versus L7. A layer-4 balancer picks a backend once per connection, which pins clients to whichever backends they first reached; with few, permanent connections that is a permanent imbalance. The mechanism and its fixes are covered in @@NET@@. Bidi makes it worse only in degree — the connections live even longer, so the imbalance never self-corrects.
Head-of-line blocking. HTTP/2 multiplexing removes request-level head-of-line blocking, so a slow stream no longer stalls the ones behind it at the application layer. It cannot remove it at the transport layer: all streams ride one TCP connection, so a single lost segment stalls delivery for every stream on that connection until it is retransmitted. On a lossy mobile link that is the dominant tail-latency source, and the only real fixes are fewer streams per connection or a transport with independent stream delivery. See @@MUX@@ for the multiplexing model in general.
When a bidi stream is the wrong shape
Only one side has payload. If the client sends one request and then only listens, use server-streaming. You get the same push semantics with half the concurrency hazard, since there is no write half to deadlock against.
Messages are rare. A stream that idles for minutes at a time still costs a stream slot against MAX_CONCURRENT_STREAMS, a keepalive budget, and a share of the connection window. At low message rates, unary calls or polling are cheaper and far easier to load-balance.
You need durability, replay or fanout. A bidi stream is a transport, not a queue: no persistence, no offsets, no consumer groups, nothing survives a process restart. If the requirement is "every message is eventually delivered even if the consumer is down for ten minutes", that is a broker, with the stream reduced to the last hop.
The client is a browser. gRPC-Web cannot express bidirectional streaming at all, because browsers expose neither HTTP/2 framing nor trailers to JavaScript. Unary and server-streaming work through a translating proxy; bidi does not. Use WebSocket or WebTransport there, or a server-streaming call paired with unary uploads, which is the shape that gRPC-Web can actually carry.
What is left is the real use case: a long-lived, stateful conversation between two services you control, where both sides emit spontaneously and ordering within each direction matters — telemetry with server-driven config pushes, agent control planes, live inference sessions.