Why architecture matters here
Signaling failures are disproportionately visible. Users forgive a second of pixelated video (a media-plane degradation), but 'the call never connected' or 'I clicked join and the spinner never stopped' — signaling failures — are the reviews that kill a product. Time-to-connected is a funnel: auth, room join, offer, answer, ICE exchange, DTLS handshake; every signaling hop adds latency and a failure opportunity before the user hears anything. Architecture determines both the depth of that funnel and how observably it fails.
The reliability requirements are stricter than the media path's. Media tolerates loss by design — codecs conceal, FEC recovers. Signaling tolerates none: a lost answer means no call; a reordered pair of offers deadlocks negotiation; a duplicated join creates a ghost participant that haunts the roster. So the channel needs ordering, exactly-once effects (idempotency), and durable session state — properties a bare WebSocket does not provide, and that must be engineered on top of it.
Scale asymmetry shapes the design too. A signaling server handles thousands of mostly-idle connections that burst on join events — a 10,000-person webinar produces a signaling stampede at the scheduled start, then near silence. Media servers meanwhile run hot continuously. Coupling them (the classic 'the SFU is also the signaling server' shortcut) means media load can starve control traffic and signaling bursts can degrade media; separating them lets each scale on its own curve, and lets signaling nodes deploy daily while media nodes drain over hours.
The architecture: every piece explained
WebSocket edge workers, stateless by design. Clients hold a single WebSocket (or WebTransport) to a signaling worker. The worker owns no authoritative state: every message it needs to route or answer is resolved against the session state store — typically Redis — which holds rooms, membership, per-member epochs, and connection-to-node mappings, with TTLs so crashed clients age out. Statelessness is what makes deploys and node failures boring: any client can reconnect to any worker and resume.
Pub/sub backbone. Peers in one room land on different workers, so workers route messages to each other through a pub/sub layer (Redis pub/sub, NATS, or Kafka for durability) keyed by room or member ID. The backbone's delivery semantics leak directly into your protocol: Redis pub/sub drops messages for disconnected subscribers, which is precisely why the protocol layer above must support resync rather than assuming delivery.
The negotiation protocol. On join, a client authenticates (short-lived JWT from the room service), receives the current roster, and negotiation begins: an SDP offer describing proposed media sections, an answer accepting or narrowing them, then trickled ICE candidates flowing both ways as STUN/TURN discovery yields host, reflexive, and relay addresses. Because both sides can decide to renegotiate simultaneously (both add a track at once), offers can cross in flight — glare. The perfect negotiation pattern resolves it by assigning asymmetric roles: the polite peer rolls back its own offer when one collides, the impolite peer ignores the incoming one; the pair converges without locks or extra round trips.
Session epochs and idempotency. Every message carries a session epoch (incremented on each reconnect) and a per-sender sequence number. Workers discard messages from stale epochs — the fix for the classic bug where a zombie connection's late messages corrupt a fresh session. Joins, leaves, and subscription changes are idempotent operations on the state store, so client retries are safe. SFU control plane: in an SFU topology, signaling also carries publish/subscribe intent ('subscribe me to Alice at 360p'), which the SFU control plane translates into media-plane forwarding decisions — keeping bandwidth policy in the control channel where it can be authenticated and audited.
End-to-end flow
Walk a two-person call end to end. Alice's client fetches a join token from the room service (HTTP, cacheable, rate-limited), opens a WebSocket to a signaling worker, and sends join(room, token, epoch=1). The worker validates the token, registers her in Redis with a TTL heartbeat, subscribes to the room's pub/sub channel, and returns the roster: Bob is already present. Alice's client creates a PeerConnection, adds her tracks, and generates an offer; simultaneously ICE gathering starts — host candidates immediately, server-reflexive from STUN in ~50ms, relay from TURN allocated as a fallback.
The offer travels: Alice → her worker → pub/sub → Bob's worker → Bob. Bob applies it as his remote description, answers, and the answer flows back along the reverse path. Candidates trickle in parallel — neither side waits for gathering to complete, which is what keeps time-to-connected near one RTT plus STUN time instead of several seconds. As candidate pairs are checked, a host-to-reflexive pair succeeds; DTLS handshakes over it; SRTP flows. Signaling then goes quiet: the media path is peer-to-peer (or peer-to-SFU) and owes the signaling plane nothing.
Mid-call, Bob's laptop switches from Wi-Fi to ethernet. His WebSocket drops; the media keeps flowing for now because its lifetime is independent. Bob's client reconnects to a different worker (the old one is mid-deploy), presents its resume token with epoch=2, and the worker rebuilds his subscriptions from Redis and replays the roster delta he missed. When his network change breaks the media path too, ICE restart runs over the restored signaling channel — a fresh candidate exchange without tearing down the session. When Alice hangs up, her leave propagates, TTLs would have caught it anyway, and the room empties. Every step is either idempotent or epoch-guarded, so any message replayed or delivered late is harmless.