READ BYTES · FRAME EVENTS · CONSUME

Streaming RPC body: queue pressure and message framing

Fragment real encoded bytes, inspect the parser buffer and follow complete events into an ordered text consumer.

60-second inspection · actual local parser

Text source and visible output

Appended output

Unparsed buffer: first 32 bytes in hex

Your experiment

Increase consumer processing time and reduce queued-event capacity. Observe paused application reads, then increase the queue and compare peak buffering. This is not HTTP/2 flow-control emulation.

The text is split into six-code-point deltas, numbered from 1. A source fault targets event 3, or the last event when fewer than three exist. Output is appended only after a complete frame is parsed and consumer processing finishes.

In streaming mode a full application queue pauses reads; one additional event may be processing. In buffered mode all body bytes arrive before parsing begins. The same 60-second inspection maps proportionally to each scenario’s modeled duration; it is not a network performance measurement.

Recent parser and consumer events

Original event fixtures

Offline byte-framing lesson. SSE parsing supports only the generated LF-delimited id/data subset; this is not a complete EventSource implementation. The alternative uses a zero compression byte plus four-byte big-endian payload length and UTF-8 JSON, not Protobuf. No HTTP/2 frames, headers, trailers, grpc-status, compression, TLS, provider API, A2A envelope, retries or reconnect/resume are implemented. The queue policy is application backpressure, not transport flow control. Duplicate suppression and strict contiguous event IDs are explicit application policies, not guarantees supplied by SSE/gRPC. Missing IDs are rejected without reordering or replay. Empty source is complete with no output; an incomplete frame remains truncated. SSE specification · gRPC message framing.

Trace queue pressure from consumer to reader

This lab follows a bounded application queue between a real local frame decoder and a sequential consumer. Only complete records enter that queue. One additional event may be active in the consumer, so capacity one means one waiting record plus one being processed, not one record total. When incremental mode finds the waiting queue full, the byte reader pauses. The pending-byte buffer can still contain data already read, showing why a count of queued events is not a direct bound on every byte retained in memory.

Control the byte and application boundaries

Local text supplies the payload, with the browser limiting entry to one hundred twenty UTF-16 code units. The model groups Unicode code points into records of up to six, each with a sequence identifier and delta string. Framing format selects the LF-delimited SSE subset or an uncompressed five-byte gRPC-style prefix around JSON. Application consumption either parses complete events incrementally or waits for the full body. Bytes per read changes fragmentation, Read interval changes scheduled arrival spacing, Processing per event changes consumer time, and Queued events bounds waiting records separately from the one active record. Parameter edits rebuild the scenario and start inspection unless reduced motion is requested.

Make the slow consumer visible

Select gRPC-style framing, incremental consumption, sixty-four bytes per read, one hundred milliseconds between reads, one thousand milliseconds per event and one queued event. Inspect until the reader shows PAUSED. The active consumer completes its event, freeing progress through the decoder and queue; later reads can resume. Reduce Processing per event and compare blocked time. Increase queue capacity and inspect the trade-off between waiting records and reader pauses. Changing capacity does not make individual consumer processing faster, and there is no second consumer hidden behind the display.

Inject failures and inspect safely

Source event fault repeats, omits or corrupts the third event, or the last available event for a shorter source; truncation removes the final three wire bytes. Ignore already-applied event IDs affects duplicate records only. Missing or malformed records can leave a sequence gap, and later records are rejected rather than reordered or replayed. Play and Pause inspect sixty seconds of viewing time; Next event advances a recorded scheduler transition; Restart inspection rewinds; Speed adjusts viewing pace; Inspection time scrubs. Visible output, pending-byte hex and recent events reflect the inspected prefix. The complete-run preview and original event list describe the full scenario, not work already consumed.

Do not confuse application pressure with transport credit

This fixture pauses an application reader; it does not implement HTTP/2 stream or connection window updates, socket-buffer limits, reactive-stream demand messages or a real gRPC client. Full-body mode deliberately bypasses that incremental read gate while receiving the body. Duplicate suppression belongs to the consumer sequence policy, not a promise of exactly-once delivery from gRPC. No retry or resume protocol repairs a missing record. Use the trace to identify where memory and waiting accumulate, then validate the corresponding behavior with a real transport before calling an integration release-ready.

Views are my own and do not represent any employer, past or present.

Search