gRPC Architecture in Depth

Why architecture matters here

Rolling your own RPC is a mistake most teams make once: framing, schema evolution, streaming, cancellation, load balancing and retry-under-deadline are each subtly hard, and gRPC is a widely-deployed implementation of all of them behind a generated, language-native API.

The wire savings are real but not a fixed ratio — protobuf writes field numbers rather than names, varint-encodes integers and needs no quoting or delimiters, so the gap against JSON is wide for many small numeric fields and narrow for a payload that is mostly free text. HTTP/2 multiplexing collapses N connections into one, which is the source of the efficiency and, as the load balancing section shows, of the most common way gRPC deployments go wrong. The rest — propagating deadlines, one place for auth and tracing, retry and balancing policy as configuration — is why you take the framework instead of putting protobuf on plain HTTP yourself.

The HTTP/2 foundation, and what gRPC adds

gRPC is not a transport. It is a calling convention layered on HTTP/2, and most of its operational surprises are an HTTP/2 mechanism showing through.

HTTP/2 supplies four things. Binary framing: HEADERS, DATA, SETTINGS, PING, WINDOW_UPDATE, RST_STREAM and GOAWAY frames, each tagged with a stream ID. Streams: an independently-terminated lane, which is what one RPC actually is — cancelling a call is a RST_STREAM on its stream, and MAX_CONCURRENT_STREAMS caps how many run at once per connection. HPACK: header compression against a shared dynamic table, so the :path, content-type and auth headers repeated on every call cost a few bytes after the first. Flow control: a credit window per stream and per connection, initially 65,535 bytes, replenished by WINDOW_UPDATE, so a slow reader stops the writer instead of buffering without bound — mature implementations grow that window from a bandwidth-delay estimate rather than leaving it at the spec default.

What gRPC adds is what HTTP never had: a method naming scheme, message framing inside the byte stream, a machine-readable status independent of the HTTP status, a deadline that travels with the request, and a schema.

The wire format: length-prefixed frames and trailing status

A unary call is one HTTP/2 stream carrying a POST. The request HEADERS frame holds :method POST and :path /pkg.Service/Method — the fully-qualified proto name, which is why gRPC has no URL design debate — plus content-type: application/grpc+proto, te: trailers, and any metadata.

DATA frames carry length-prefixed messages. Each message is preceded by exactly five bytes: one compressed-flag byte (0 = identity, 1 = compressed with the algorithm named in grpc-encoding), then a 4-byte big-endian length. Because of that prefix a message boundary never depends on a frame boundary — a 6 MB message spans many DATA frames, several tiny messages share one — and the common 4 MiB receive limit is enforceable before any payload is parsed.

Then the part that trips people up: the status arrives in trailers. The response ends with a second HEADERS frame carrying grpc-status and optionally grpc-message. It has to be trailers, because on a streaming call the server cannot know whether the RPC succeeded until after the last message is written, and even a unary handler can fail after headers are flushed. The HTTP status is 200 on a failed RPC: the transport worked, the call did not. A server failing immediately sends a Trailers-Only response — one HEADERS frame with END_STREAM carrying both statuses, no DATA at all.

The architecture: every piece explained

Read the diagram top to bottom. The top row is one RPC in flight: a stub generated from your .proto, a stream multiplexed onto a shared HTTP/2 connection, a generated dispatcher on the far side. The middle rows are what rides on it — the schema, the interceptor chain, and the four call shapes the streaming model allows. The bottom rows are the operational surface: finding backends and spreading load across them, bounding the whole call tree with a deadline, and the bridges browsers and REST clients need. Each gets its own section below.

Four call types and what they really guarantee

The four shapes differ only in which side may send more than one message. Unary: one request, one response. Server-streaming: the client sends one message and half-closes, the server writes many. Client-streaming: the reverse. Bidirectional: both write independently, and the interleaving is entirely the application's business — a bidi stream is not request/response unless you make it so.

What you get: messages on one stream are ordered and never lost without the stream erroring, since it is all one TCP byte stream. What you do not get is any guarantee once the stream breaks. A server stream that dies at message 400 of 1000 is indistinguishable, to the client, from a server that meant to send 400 — unless your protocol carries a completion marker or a resumption token. Streams are not queues either: no persistence, no replay, no consumer groups. The only real backpressure is the HTTP/2 window, and it helps only if your binding exposes manual flow control instead of draining into an unbounded application buffer.

Protobuf as the IDL, and the evolution rules that hold

The .proto file is the contract; codegen is convenience on top. On the wire a message is key-value pairs where the key is a varint packing the field number and wire type, not the name. Field 1 with varint type is tag byte 0x08. That one choice explains both why protobuf is small and why the compatibility rules are what they are.

Safe: adding a field with a fresh number — old readers keep it as an unknown field and most runtimes preserve it through a parse/serialize round trip, which is what stops a service in the middle from silently stripping data; renaming a field; adding an enum value if readers tolerate unknown ones.

Unsafe: reusing a number after deleting a field, which deserializes old data into the wrong field — hence reserved 4, 7; and reserved "email";; changing a type across wire-type families; changing repeated to singular. Mind the defaults too: proto3 scalars have no presence unless declared optional, so an unset int32 and a deliberate zero are identical bytes, and enum value 0 is what every old reader sees for a value it does not know.

Deadlines, cancellation, and metadata

A deadline is not a client-side timer that gives up quietly. It is a header — grpc-timeout, a value plus a unit suffix (H, M, S, m, u, n), so 100m is 100 milliseconds — so the server knows how long it has. Bindings expose the time remaining on the request context, and the idiom that makes this valuable is propagation: a handler passes that context downstream, the binding recomputes what is left and re-encodes it on the outgoing call, so a budget set once at the edge shrinks monotonically down the whole call tree. Cancellation rides the same rails: a client that gives up sends RST_STREAM, the server's context is cancelled, and its own in-flight downstream calls are cancelled in turn — real work saved, but only if handlers check cancellation between stages.

Metadata is gRPC's name for HTTP/2 headers and carries everything not modelled in the proto: tokens, trace context, tenant IDs. Keys are lowercase, grpc- is reserved, and a -bin suffix means arbitrary bytes, base64-encoded on the wire. Interceptors wrap calls on both sides for auth, tracing, metrics and panic recovery. The sharp edge: unary and streaming interceptors are separate types in most bindings, so a chain wired only for unary silently stops applying the day someone adds a streaming method — a common way for auth to grow a hole.

End-to-end call flow

Trace stub.GetUser(GetUserRequest{id: 42}) with a one-second deadline.

Client interceptors run first: auth puts a bearer token in metadata, tracing injects trace context. The LB policy picks a subchannel and the channel opens a new HTTP/2 stream on that existing connection — no TCP or TLS handshake. The HEADERS frame carries :path /pkg.UserService/GetUser, content-type: application/grpc+proto, te: trailers, grpc-timeout: 1S and the metadata, HPACK-compressed against the connection's dynamic table.

The body is small enough to spell out. Field 1 with varint type is tag 0x08; 42 is 0x2a. The message is two bytes, so the DATA frame carries seven: 00 00 00 00 02 08 2a — flag 0, big-endian length 2, then the message.

The server's generated dispatcher routes on :path, server interceptors run, and the handler receives a context whose deadline is already ticking; its database call and any downstream RPC inherit what is left. Response headers go out, the User message follows as another length-prefixed payload, and a final HEADERS frame carries grpc-status: 0 with END_STREAM. Make it server-streaming instead — token-by-token LLM output, say — and the only change is many length-prefixed messages before that trailing frame, interleaved by HTTP/2 with every other stream, with per-stream flow control stopping a slow consumer from letting the server run ahead.

Name resolution and client-side load balancing

A channel is not a connection. The target string selects a resolver by scheme — dns:///svc.ns.svc.cluster.local:50051, unix:, xds:/// — the resolver returns a set of addresses, and the LB policy decides how many subchannels (one TCP+HTTP/2 connection each) to open and which one each RPC uses. Re-resolution is rate-limited to avoid hammering DNS, so endpoint churn is not observed instantly.

pick_first tries addresses in order and keeps a single connection; it is the default, so an unconfigured channel talks to exactly one backend. round_robin connects to every resolved address and rotates per RPC, and needs a resolver that returns all endpoints — in Kubernetes a headless service, not a ClusterIP. Beyond that is lookaside balancing, today xDS, with the gRPC client itself speaking LDS/RDS/CDS/EDS: proxyless mesh, no sidecar in the data path. For the sidecar version see service mesh architecture.

Why an L4 proxy breaks gRPC load distribution

An L4 balancer picks a backend once, at connection setup, and every byte afterwards follows that choice. HTTP/1.1 tolerated this because connections were many and short. gRPC connections are few and effectively permanent, with all the concurrency in streams inside one connection — so clients behind a ClusterIP pin themselves to whichever backends they first landed on, and scaling to thirty pods sends traffic to none of the twenty new ones. The fixes are always the same three: balance in the client, put a real L7 proxy in the path, or use xDS. Classic connection-pool sizing math does not transfer either — extra connections buy nothing MAX_CONCURRENT_STREAMS was not already granting.

Keepalives, connection age, and retries

A long-lived HTTP/2 connection with no traffic is invisible to both ends until something needs it: NAT tables, cloud load balancer idle timeouts and stateful firewalls drop the flow silently, and the next RPC finds out only after a TCP retransmission timeout. Hence HTTP/2 PING keepalives. Servers police them: enforcement sets a minimum ping interval and by default refuses pings on a connection with no active streams, so a client pinging too often gets a GOAWAY carrying too_many_pings, reconnects, and does it again — a reconnect storm caused purely by the two ends disagreeing. The deliberate version of the same mechanism is a server-side maximum connection age plus grace: GOAWAY, let streams below the last-stream-ID finish, close, and let clients re-resolve. That is the standard way to rebalance a fleet after a scale-out.

Retry policy is configuration, not code. The service config is JSON, delivered by the resolver (a DNS TXT record under _grpc_config, or xDS) or set as the client default:

{"methodConfig": [{
   "name": [{"service": "pkg.UserService", "method": "GetUser"}],
   "timeout": "1s",
   "retryPolicy": {
     "maxAttempts": 4, "initialBackoff": "0.1s", "maxBackoff": "1s",
     "backoffMultiplier": 2,
     "retryableStatusCodes": ["UNAVAILABLE", "RESOURCE_EXHAUSTED"]}}],
 "retryThrottling": {"maxTokens": 100, "tokenRatio": 0.1}}

Three details decide whether it helps or hurts. Commitment: once the first response message reaches the client the RPC is committed and cannot be retried, which is why streaming responses are effectively unretryable after they start. Throttling: retryThrottling is a token bucket per server — failures spend tokens, successes refill them — and is what stops a retry policy turning a brownout into an outage. Server participation: an overloaded server returns grpc-retry-pushback-ms to dictate or forbid the next attempt, and every attempt carries grpc-previous-rpc-attempts. Hedging inverts the idea: fire a second attempt after hedgingDelay and take the first answer — a tail-latency tool, safe only on idempotent methods, since every hedged attempt may execute.

Status codes, credentials, and gRPC-Web

gRPC defines a closed set of 17 status codes, and using them precisely is what makes generic retry and alerting possible. UNAVAILABLE (14) means the RPC did not reach a working server and is safe to retry; DEADLINE_EXCEEDED (4) means you stopped waiting and the work may well have happened; FAILED_PRECONDITION (9) says do not retry until state changes, while ABORTED (10) says retry after a concurrency conflict; RESOURCE_EXHAUSTED (8) is quota or message size; UNIMPLEMENTED (12) is a genuine capability probe; INTERNAL (13) is a server bug and UNKNOWN (2) an exception nobody mapped. Returning INTERNAL for everything throws the model away. Structured errors belong in a google.rpc.Status with typed Any details in grpc-status-details-bin, not a second format smuggled into grpc-message.

Security splits in two. Channel credentials secure the connection: TLS over ALPN-negotiated h2, or mutual TLS where the client certificate is the service identity (see mTLS). Call credentials attach a per-RPC token to metadata and are refreshed by the credential plugin rather than by your interceptor. They compose, and gRPC deliberately refuses to send call credentials over an insecure channel, because a bearer token on plaintext h2c is a token you have given away.

Browsers cannot speak gRPC at all — not because of protobuf, but because no browser API exposes HTTP/2 framing or lets JavaScript read trailers, and the status lives in trailers. gRPC-Web moves the trailers into the response body as one more length-prefixed frame, flagged by setting the high bit of the flag byte, so an ordinary fetch can parse it. Something has to perform that re-encoding, which is why gRPC-Web needs a proxy, and why client-streaming and bidirectional calls are unavailable to browsers while unary and server-streaming work. grpc-gateway is the neighbouring bridge, transcoding JSON/REST from google.api.http annotations. gRPC over HTTP/3 exists, but HTTP/2 remains the deployed transport — see HTTP/3 and QUIC.