Video CDN Architecture in Depth

Why architecture matters here

A video CDN is not a general-purpose CDN with bigger disks. The underlying machinery is the same - PoP catchment, cache keys, TTL layering, invalidation, origin shielding, TLS termination - and that machinery is covered in Content Delivery Network Architecture in Depth. What changes is the shape of the workload it is applied to, and the shape is unusual enough that several defaults which are correct for a website are actively wrong for video.

Three properties cause most of the divergence. One title is thousands of objects. Every rung of the bitrate ladder, times every segment, times every audio and subtitle track, times every codec and DRM variant. A cache holding one HTML page holds one object; a cache holding one two-hour film may hold thirty thousand. The bytes and the requests point in opposite directions. A manifest is a few kilobytes fetched every few seconds by every viewer; a segment is a few megabytes fetched once. Any policy tuned on request counts optimises the manifest and ignores the bill. Live and VOD share almost nothing at the cache. A VOD object accumulates reuse over weeks. A live segment is requested by the entire concurrent audience inside a window a few seconds wide and is then dead forever - reuse is concurrency, not longevity, and no TTL setting changes that.

Everything below follows from those three. Where a neighbouring subtopic already has its own article - packaging, ad insertion, DRM, ABR logic, steering - this one stops at the delivery boundary and links out rather than re-explaining it.

The architecture: every layer explained

The diagram below is the delivery path end to end. Read it as four bands; each band has a dedicated article behind it, so the legend here is deliberately one clause per box.

Request path. Player, edge PoP, mid-tier shield. The player is hls.js, dash.js, Shaka or a native platform player, and it - not the server - chooses which rendition to request for each segment (see adaptive bitrate).

Origination. Just-in-time packaging turns one CMAF mezzanine into HLS or DASH at request time instead of storing every combination (see just-in-time packaging), and server-side ad insertion rewrites the manifest per viewer (see SSAI).

Access control. Signed URLs and tokens gate every segment at the edge; DRM licensing runs on a separate path the CDN does not serve (see video DRM).

Steering and telemetry. Player beacons feed per-CDN QoE measurement, which feeds multi-CDN pathway selection - the full control loop, its TTL and its failover rules are covered in content steering and are not repeated here.

The manifest is not just another cached object

Split the cache policy at the manifest boundary before tuning anything else. The manifest and the segments it points at have opposite freshness requirements, and the most common video CDN configuration error is applying one rule to both.

Segments are effectively immutable. Once a packager emits seg_00412.m4s for a given rendition, those bytes never change; a re-encode produces a new path. So segments take the treatment general CDNs reserve for hashed build assets: a very long TTL, and no purge workflow at all. For VOD, max-age=31536000, immutable is correct. For live, the right ceiling is the DVR window, not infinity - a segment that has fallen off the sliding playlist will never be requested again, and holding it just wastes admission slots that hot objects want.

The live media playlist is the opposite. It mutates every segment duration, and its whole value is being current. Its shared TTL has to sit below the target duration - one or two seconds against a four-second target is typical - because a playlist one generation behind hides the newest segment from every player that reads it.

That is where the general advice inverts. In an ordinary CDN configuration stale-while-revalidate is the highest-leverage directive there is, because nobody should wait on a refill. On a live media playlist it is harmful: serving a stale playlist instantly means the player learns about segment N a beat late, its buffer drains by that much, and it settles a fixed distance further from the live edge - permanently, because every subsequent reload repeats the offset. Live playlists want fresh or nothing. stale-if-error is still worth having, narrowly scoped, so a packager blip does not empty every buffer at once.

The manifest is also the hot object by request count. With a four-second target duration a live player reloads its media playlist roughly every two to four seconds, so half a million concurrent viewers generate on the order of 125k-250k playlist requests per second against about 125k segment requests per second - similar rates for objects that differ in size by three orders of magnitude. Keep manifests in the memory tier and segments on SSD; they are different workloads sharing a hostname.

One consequence for personalisation: if SSAI or a per-viewer token is baked into the manifest, that manifest is a private object and will not be shared by anyone. That is fine and intended - but it means the packager tier, not the cache, absorbs the playlist request rate above. Keep the personalisation in the manifest and keep the segments byte-identical for everyone, or the object explosion in the next section gets multiplied by the audience size.

The ABR ladder multiplies the cacheable object count

Do the arithmetic once and the caching strategy explains itself. Take a two-hour title, a six-rung bitrate ladder, and four-second segments.

video segments   7200s / 4s = 1800 per rung  x 6 rungs        = 10,800
audio renditions 1800 x (5 languages x 2 codecs = 10)      = 18,000
subtitle (segmented WebVTT/IMSC), 5 languages              =  9,000
init segments, one per rendition                           =     21
                                                             ------
one title, one codec family, one DRM system                = ~37,800 objects

Then apply the multipliers. Shipping H.264 for legacy devices alongside HEVC or AV1 for modern ones doubles or triples the video count (see H.264 vs H.265 vs AV1). DRM is the sharpest single decision: CENC common encryption lets one encrypted copy serve Widevine and PlayReady, but a separate Apple packaging path means a second full set of video and audio segments. A catalogue of ten thousand titles at this shape is on the order of 10⁸-10⁹ objects. No edge PoP holds a meaningful fraction of that, and the interesting question stops being "how big is the disk".

The working set is titles times rungs, not titles

Popularity is Zipf-shaped across titles, and it is also shaped across rungs. Most viewing sits on two or three middle rungs; the top rung is delivered only to viewers with the bandwidth and the screen to justify it, and the bottom rungs only to viewers who fell there. Both ends must exist and both are cold almost everywhere. So a hot title still generates a steady stream of misses on its unpopular rungs, and each of those misses is a multi-megabyte origin or shield fetch rather than a few kilobytes.

This is why byte hit ratio and request hit ratio diverge so violently for video, and why the top rung deserves separate attention: for the same wall-clock second it carries five to eight times the bytes of a middle rung, so a small percentage of top-rung requests can be a large percentage of egress. Track delivered bytes per rung, not just per title.

Admission policy beats disk size

The default behaviour - cache everything on first request - is wrong at this object count. A single viewer wandering through the long tail writes thousands of segments that will never be requested again, evicting hot ones as they go. Two-touch admission (only cache on the second request within a window, tracked in a small probabilistic filter) costs one extra origin fetch per genuinely-cold object and stops the tail from churning the working set. Per-title encoding, which varies the ladder by content complexity, changes the object mix further (see per-title encoding).

Live delivery: the rolling window that defeats reuse

A VOD object earns its keep over time: cache it once, serve it for weeks. A live segment has no lifetime. Segment N is published, requested by essentially the entire concurrent audience within one segment duration, and then never requested again. The reuse factor is the number of concurrent viewers, compressed into a window as wide as the segment duration.

So for live the cache is not really caching - it is request collapsing at the moment of publication. The mechanism itself (collapsed forwarding, the uncacheable-response serialisation hazard, shield placement) is general and is covered in the CDN article. The video-specific part is that the collapse window is bounded by segment duration rather than by TTL, and that the misses arrive synchronised rather than spread out.

Phase-locking is the thing to design for. Every player reloads the media playlist on a clock derived from the same publication event, so the audience is in phase. Three hundred thousand viewers on an edge do not arrive uniformly at 75k requests per second; they arrive as a spike every four seconds with near-idle gaps between. Capacity planning against the mean under-provisions by whatever the peak-to-mean ratio turns out to be, and that ratio gets worse as segments get shorter.

The event-start storm

At kickoff the audience joins over perhaps sixty seconds, and every one of those sessions cold-fills the same short list: the multivariant manifest, the media playlist, the init segment for its chosen video rung, the init segment for its audio track, and then the first media segments of both. With six rungs, three audio tracks and two subtitle tracks that is about two dozen distinct objects that must be filled per PoP before anything is warm. The shield exists so the origin sees one fetch per object instead of one per PoP - the difference between roughly twenty-five origin fetches and twenty-five times the number of active PoPs.

Note what the numbers say about sizing. New objects appear at (rungs + audio + subtitle tracks) / segment duration - about 2.75 per second in the example above. That is nothing. The origin tier never falls over because of object rate; it falls over because of connection concurrency during a synchronised reload, which is a different resource entirely and needs to be measured as such. Ingest and the rest of the live chain are covered in live streaming architecture.

Segment duration is a cache knob, not just a latency knob

Segment duration is usually discussed as a latency parameter, but it sets three cache properties at the same time and they pull in different directions.

Request rate scales inversely. One viewer-hour is 1800 segment requests at two seconds and 600 at six, for identical bytes. At half a million concurrent viewers that is 250k versus 83k requests per second, and every one of those requests pays a token validation, a cache lookup, a log line, and its share of connection overhead. Tripling the request rate to move the same bytes is a real cost at the edge even though the bill is byte-denominated.

Object count scales inversely too, and small objects are worse per byte. Three times as many objects, each a third the size, means three times the index entries and metadata, more read amplification, and a working set that is harder to keep resident.

Encoding efficiency degrades. Each segment must begin with an IDR frame, so shorter segments mean more keyframes and a higher bitrate for the same quality - you pay for short segments twice, once in requests and once in bytes (see GOP structure).

But the collapse window shrinks as well. Halving the segment duration halves the window in which a burst of requests for the same object can be merged, while doubling how often that burst occurs. Shorter segments are strictly worse for the shield.

Against all that, shorter segments cut startup time: a player that buffers three segments before rendering waits six seconds at two-second segments and eighteen at six. Four seconds is the common settling point for VOD. For live, the modern answer is not to shorten the segment at all but to keep it and publish CMAF chunks inside it, which decouples latency from segment duration entirely (see CMAF).

Pre-positioning: filling caches before anyone asks

Launch hour is the worst possible combination: the largest audience the title will ever have, arriving against a completely cold cache. Every one of those early sessions pays an origin round trip on the manifest, the init segment and the first media segment - serially, because the player cannot request what it has not yet parsed. That lands directly on join time, which is the metric abandonment tracks.

Pre-positioning fixes it by pushing bytes into edge and shield caches before the release timestamp, using a CDN fill API or a scripted prefetch pull. The discipline is in choosing what to push. Pre-positioning an entire 4K title to every PoP is a large, deliberate egress spend on a title that may be unpopular in most of them. The high-value subset is small:

The manifests and every init segment. Kilobytes, needed by every single joiner, and on the critical path before anything else can be requested.

The first sixty to ninety seconds of the two or three middle rungs. That covers join time and the early ABR ramp. The rest of the title fills naturally as the audience watches it, which is a conveniently self-rate-limiting fill: viewers progress through a title at 1x, so the origin sees a smooth pull rather than a spike.

Ad creatives, for anything ad-supported. This is the sharpest case. An ad break is served to the whole audience within seconds of each other, so an unfilled creative produces a synchronised origin storm identical in shape to a live segment storm - except it is triggered by the ad decision server rather than by the packager, so it usually is not on the same dashboard. Pre-position creatives and slate assets with the same care as init segments.

Scope pre-positioning by region using historical demand rather than pushing everywhere, and give pre-positioned objects an explicit expiry so a title that flopped does not hold edge capacity for a month. For the long tail the better tool is hierarchy rather than prefetch: let the regional tier hold a much larger footprint and the edge hold only what is locally hot. A regional hit is not free, but tens of milliseconds beats a cross-continent origin fetch by enough to keep it off the rebuffer budget.