Video CDN Architecture in Depth

Why architecture matters here

A video CDN is not a general-purpose CDN with bigger disks. The underlying machinery is the same - PoP catchment, cache keys, TTL layering, invalidation, origin shielding, TLS termination - and that machinery is covered in Content Delivery Network Architecture in Depth. What changes is the shape of the workload it is applied to, and the shape is unusual enough that several defaults which are correct for a website are actively wrong for video.

Three properties cause most of the divergence. One title is thousands of objects. Every rung of the bitrate ladder, times every segment, times every audio and subtitle track, times every codec and DRM variant. A cache holding one HTML page holds one object; a cache holding one two-hour film may hold thirty thousand. The bytes and the requests point in opposite directions. A manifest is a few kilobytes fetched every few seconds by every viewer; a segment is a few megabytes fetched once. Any policy tuned on request counts optimises the manifest and ignores the bill. Live and VOD share almost nothing at the cache. A VOD object accumulates reuse over weeks. A live segment is requested by the entire concurrent audience inside a window a few seconds wide and is then dead forever - reuse is concurrency, not longevity, and no TTL setting changes that.

Everything below follows from those three. Where a neighbouring subtopic already has its own article - packaging, ad insertion, DRM, ABR logic, steering - this one stops at the delivery boundary and links out rather than re-explaining it.

The architecture: every layer explained

The diagram below is the delivery path end to end. Read it as four bands; each band has a dedicated article behind it, so the legend here is deliberately one clause per box.

Request path. Player, edge PoP, mid-tier shield. The player is hls.js, dash.js, Shaka or a native platform player, and it - not the server - chooses which rendition to request for each segment (see adaptive bitrate).

Origination. Just-in-time packaging turns one CMAF mezzanine into HLS or DASH at request time instead of storing every combination (see just-in-time packaging), and server-side ad insertion rewrites the manifest per viewer (see SSAI).

Access control. Signed URLs and tokens gate every segment at the edge; DRM licensing runs on a separate path the CDN does not serve (see video DRM).

Steering and telemetry. Player beacons feed per-CDN QoE measurement, which feeds multi-CDN pathway selection - the full control loop, its TTL and its failover rules are covered in content steering and are not repeated here.

The manifest is not just another cached object

Split the cache policy at the manifest boundary before tuning anything else. The manifest and the segments it points at have opposite freshness requirements, and the most common video CDN configuration error is applying one rule to both.

Segments are effectively immutable. Once a packager emits seg_00412.m4s for a given rendition, those bytes never change; a re-encode produces a new path. So segments take the treatment general CDNs reserve for hashed build assets: a very long TTL, and no purge workflow at all. For VOD, max-age=31536000, immutable is correct. For live, the right ceiling is the DVR window, not infinity - a segment that has fallen off the sliding playlist will never be requested again, and holding it just wastes admission slots that hot objects want.

The live media playlist is the opposite. It mutates every segment duration, and its whole value is being current. Its shared TTL has to sit below the target duration - one or two seconds against a four-second target is typical - because a playlist one generation behind hides the newest segment from every player that reads it.

That is where the general advice inverts. In an ordinary CDN configuration stale-while-revalidate is the highest-leverage directive there is, because nobody should wait on a refill. On a live media playlist it is harmful: serving a stale playlist instantly means the player learns about segment N a beat late, its buffer drains by that much, and it settles a fixed distance further from the live edge - permanently, because every subsequent reload repeats the offset. Live playlists want fresh or nothing. stale-if-error is still worth having, narrowly scoped, so a packager blip does not empty every buffer at once.

The manifest is also the hot object by request count. With a four-second target duration a live player reloads its media playlist roughly every two to four seconds, so half a million concurrent viewers generate on the order of 125k-250k playlist requests per second against about 125k segment requests per second - similar rates for objects that differ in size by three orders of magnitude. Keep manifests in the memory tier and segments on SSD; they are different workloads sharing a hostname.

One consequence for personalisation: if SSAI or a per-viewer token is baked into the manifest, that manifest is a private object and will not be shared by anyone. That is fine and intended - but it means the packager tier, not the cache, absorbs the playlist request rate above. Keep the personalisation in the manifest and keep the segments byte-identical for everyone, or the object explosion in the next section gets multiplied by the audience size.

The ABR ladder multiplies the cacheable object count

Do the arithmetic once and the caching strategy explains itself. Take a two-hour title, a six-rung bitrate ladder, and four-second segments.

video segments   7200s / 4s = 1800 per rung  x 6 rungs        = 10,800
audio renditions 1800 x (5 languages x 2 codecs = 10)      = 18,000
subtitle (segmented WebVTT/IMSC), 5 languages              =  9,000
init segments, one per rendition                           =     21
                                                             ------
one title, one codec family, one DRM system                = ~37,800 objects

Then apply the multipliers. Shipping H.264 for legacy devices alongside HEVC or AV1 for modern ones doubles or triples the video count (see H.264 vs H.265 vs AV1). DRM is the sharpest single decision: CENC common encryption lets one encrypted copy serve Widevine and PlayReady, but a separate Apple packaging path means a second full set of video and audio segments. A catalogue of ten thousand titles at this shape is on the order of 10⁸-10⁹ objects. No edge PoP holds a meaningful fraction of that, and the interesting question stops being "how big is the disk".

The working set is titles times rungs, not titles

Popularity is Zipf-shaped across titles, and it is also shaped across rungs. Most viewing sits on two or three middle rungs; the top rung is delivered only to viewers with the bandwidth and the screen to justify it, and the bottom rungs only to viewers who fell there. Both ends must exist and both are cold almost everywhere. So a hot title still generates a steady stream of misses on its unpopular rungs, and each of those misses is a multi-megabyte origin or shield fetch rather than a few kilobytes.

This is why byte hit ratio and request hit ratio diverge so violently for video, and why the top rung deserves separate attention: for the same wall-clock second it carries five to eight times the bytes of a middle rung, so a small percentage of top-rung requests can be a large percentage of egress. Track delivered bytes per rung, not just per title.

Admission policy beats disk size

The default behaviour - cache everything on first request - is wrong at this object count. A single viewer wandering through the long tail writes thousands of segments that will never be requested again, evicting hot ones as they go. Two-touch admission (only cache on the second request within a window, tracked in a small probabilistic filter) costs one extra origin fetch per genuinely-cold object and stops the tail from churning the working set. Per-title encoding, which varies the ladder by content complexity, changes the object mix further (see per-title encoding).

Live delivery: the rolling window that defeats reuse

A VOD object earns its keep over time: cache it once, serve it for weeks. A live segment has no lifetime. Segment N is published, requested by essentially the entire concurrent audience within one segment duration, and then never requested again. The reuse factor is the number of concurrent viewers, compressed into a window as wide as the segment duration.

So for live the cache is not really caching - it is request collapsing at the moment of publication. The mechanism itself (collapsed forwarding, the uncacheable-response serialisation hazard, shield placement) is general and is covered in the CDN article. The video-specific part is that the collapse window is bounded by segment duration rather than by TTL, and that the misses arrive synchronised rather than spread out.

Phase-locking is the thing to design for. Every player reloads the media playlist on a clock derived from the same publication event, so the audience is in phase. Three hundred thousand viewers on an edge do not arrive uniformly at 75k requests per second; they arrive as a spike every four seconds with near-idle gaps between. Capacity planning against the mean under-provisions by whatever the peak-to-mean ratio turns out to be, and that ratio gets worse as segments get shorter.

The event-start storm

At kickoff the audience joins over perhaps sixty seconds, and every one of those sessions cold-fills the same short list: the multivariant manifest, the media playlist, the init segment for its chosen video rung, the init segment for its audio track, and then the first media segments of both. With six rungs, three audio tracks and two subtitle tracks that is about two dozen distinct objects that must be filled per PoP before anything is warm. The shield exists so the origin sees one fetch per object instead of one per PoP - the difference between roughly twenty-five origin fetches and twenty-five times the number of active PoPs.

Note what the numbers say about sizing. New objects appear at (rungs + audio + subtitle tracks) / segment duration - about 2.75 per second in the example above. That is nothing. The origin tier never falls over because of object rate; it falls over because of connection concurrency during a synchronised reload, which is a different resource entirely and needs to be measured as such. Ingest and the rest of the live chain are covered in live streaming architecture.

Segment duration is a cache knob, not just a latency knob

Segment duration is usually discussed as a latency parameter, but it sets three cache properties at the same time and they pull in different directions.

Request rate scales inversely. One viewer-hour is 1800 segment requests at two seconds and 600 at six, for identical bytes. At half a million concurrent viewers that is 250k versus 83k requests per second, and every one of those requests pays a token validation, a cache lookup, a log line, and its share of connection overhead. Tripling the request rate to move the same bytes is a real cost at the edge even though the bill is byte-denominated.

Object count scales inversely too, and small objects are worse per byte. Three times as many objects, each a third the size, means three times the index entries and metadata, more read amplification, and a working set that is harder to keep resident.

Encoding efficiency degrades. Each segment must begin with an IDR frame, so shorter segments mean more keyframes and a higher bitrate for the same quality - you pay for short segments twice, once in requests and once in bytes (see GOP structure).

But the collapse window shrinks as well. Halving the segment duration halves the window in which a burst of requests for the same object can be merged, while doubling how often that burst occurs. Shorter segments are strictly worse for the shield.

Against all that, shorter segments cut startup time: a player that buffers three segments before rendering waits six seconds at two-second segments and eighteen at six. Four seconds is the common settling point for VOD. For live, the modern answer is not to shorten the segment at all but to keep it and publish CMAF chunks inside it, which decouples latency from segment duration entirely (see CMAF).

Pre-positioning: filling caches before anyone asks

Launch hour is the worst possible combination: the largest audience the title will ever have, arriving against a completely cold cache. Every one of those early sessions pays an origin round trip on the manifest, the init segment and the first media segment - serially, because the player cannot request what it has not yet parsed. That lands directly on join time, which is the metric abandonment tracks.

Pre-positioning fixes it by pushing bytes into edge and shield caches before the release timestamp, using a CDN fill API or a scripted prefetch pull. The discipline is in choosing what to push. Pre-positioning an entire 4K title to every PoP is a large, deliberate egress spend on a title that may be unpopular in most of them. The high-value subset is small:

The manifests and every init segment. Kilobytes, needed by every single joiner, and on the critical path before anything else can be requested.

The first sixty to ninety seconds of the two or three middle rungs. That covers join time and the early ABR ramp. The rest of the title fills naturally as the audience watches it, which is a conveniently self-rate-limiting fill: viewers progress through a title at 1x, so the origin sees a smooth pull rather than a spike.

Ad creatives, for anything ad-supported. This is the sharpest case. An ad break is served to the whole audience within seconds of each other, so an unfilled creative produces a synchronised origin storm identical in shape to a live segment storm - except it is triggered by the ad decision server rather than by the packager, so it usually is not on the same dashboard. Pre-position creatives and slate assets with the same care as init segments.

Scope pre-positioning by region using historical demand rather than pushing everywhere, and give pre-positioned objects an explicit expiry so a title that flopped does not hold edge capacity for a month. For the long tail the better tool is hierarchy rather than prefetch: let the regional tier hold a much larger footprint and the edge hold only what is locally hot. A regional hit is not free, but tens of milliseconds beats a cross-continent origin fetch by enough to keep it off the rebuffer budget.

Low-latency delivery through an intermediary cache

LL-HLS and LL-DASH publish partial segments or CMAF chunks before the enclosing segment is complete. The encoder, packager and player-side control loop are covered in low-latency live streaming. What matters here is what the change does to every cache between the packager and the player, because an intermediary that behaves normally destroys the entire latency benefit.

The object is being written while it is being read. A proxy that buffers the complete response before forwarding it - which is the default in most of them - holds the first chunk until the last chunk arrives, reintroducing exactly the segment-duration delay the architecture removed. The edge has to relay chunked transfer bytes as they arrive. Concretely that means response buffering off on the segment path, and a cache that can store and serve partial objects so a second requester arriving mid-write attaches to the in-progress fill instead of opening its own origin fetch. Without the second property, a thousand viewers arriving during a chunked segment produce a thousand concurrent origin streams.

Blocking playlist reload converts request rate into held connections. An LL-HLS player asks for a playlist with _HLS_msn and _HLS_part parameters naming a part that does not exist yet, and the server holds the request open until it does. A quarter of a million viewers is now a quarter of a million long-lived connections per region rather than a burst of short ones. That is a file-descriptor and memory ceiling, not a bandwidth one, and it is usually what actually caps LL-HLS scale. It also means every held request releases at the same instant, so the follow-on segment burst is even more tightly phase-locked than in normal live.

Validate this per PoP, not globally. A single edge that quietly buffers whole segments produces a cohort of viewers running three or four times the latency with zero errors and a perfectly healthy hit ratio. Latency has to be measured per edge or that failure is invisible.

Per-segment authorization: tokens, geo, and DRM

Every segment request is an independent authorization decision. At four-second segments a single viewer generates 900 of them per hour; half a million concurrent viewers is roughly 125 authorization checks per millisecond. That budget rules out any backend call - it has to be arithmetic the edge can do locally.

The standard construction is an HMAC over a path prefix plus an expiry, verified at the edge against a shared secret. Two rules decide whether it works.

The token must be excluded from the cache key. If it is not, every viewer mints a private copy of every segment, the object count is multiplied by the audience, and offload collapses to zero while the origin serves the entire event. This is the single most common video CDN misconfiguration, and it presents as a mysteriously enormous bill rather than as an error. Strip the token parameter from the key and validate it before the lookup.

The token must be scoped to a path prefix, not a URL. A per-URL token means minting 1800 tokens per rung per session. Scope it to the title or session directory, and scope the expiry to something that outlives a realistic session or the player will hit a 403 ninety minutes into a film. If tokens must be short-lived, refresh them on the manifest reload - which is another reason the manifest is the per-viewer object and the segments are not.

Resist binding tokens to a client IP. Mobile handoff between Wi-Fi and cellular changes the address mid-session, and CGNAT makes the address non-identifying anyway; binding to an ASN or a coarse prefix keeps most of the anti-sharing value without the false positives.

Geo and blackout rules have to be evaluated on segments as well as manifests. A manifest-only check is bypassed the moment a segment URL leaks, and segment URLs leak. Live sports blackouts are worse than static geo policy because they are per-event, per-region and can change during the event, so the decision needs to propagate to the edges within about one segment duration to be meaningful.

DRM sits deliberately outside this path. The segments are already encrypted, so the CDN serves ciphertext and needs no key material at all - which is what makes multi-CDN safe. The license exchange is per-session, must never be cached, and belongs on its own hostname; see video DRM.

The QoE metrics that actually matter

The structural point first: the metrics that decide whether video delivery worked are measured in the player, not at the edge. A CDN dashboard reports hit ratio, edge latency and status codes, and can be entirely green while a cohort of viewers rebuffers continuously. Nothing on the CDN side observes a stall. Player beacons are the primary telemetry and the CDN metrics are supporting evidence, not the other way round.

Video start time (join time). Manifest request to first frame rendered. It is a serial chain - manifest fetch, parse, init segment, first media segment, decoder setup - so a cold cache adds a round trip at each link rather than once. This is the metric pre-positioning exists to protect.

Rebuffer ratio. Stalled seconds divided by stalled plus played seconds. Report it as a distribution, never a mean: a fleet average of 0.4% comfortably hides three percent of sessions sitting at ten percent, and those sessions are a region, an ASN or a device model, not random noise.

Bitrate delivered. The time-weighted average rendition actually played. It answers a question nothing else does: whether the ladder you are paying to encode and store is being used. If the top two rungs are almost never delivered, they are pure cost.

Exit before video start. The fraction of playback attempts abandoned before the first frame appears. It is the only one of these that counts the viewers you lost rather than the ones who stayed, and it moves with join time closely enough to be its business translation.

The bridge back to infrastructure is per-segment time to first byte and download throughput, split by cache status and by PoP. That split is what distinguishes the two explanations for a rebuffer spike: more misses, or the same number of misses that got slower. A rebuffer spike with an unchanged hit ratio points at the miss path - shield, origin or backbone - not at cache efficiency.

Dimension all of it by CDN, region, ASN, device class and rendition; a global average hides every failure that matters. Those per-CDN aggregates are also the input to multi-CDN pathway selection, which is covered in content steering.