Content Delivery Network Architecture in Depth

Why architecture matters here

The number a CDN is sold on - how many points of presence it operates - is the least interesting one. The number that decides whether the deployment worked is origin offload: the fraction of bytes users received that your servers never sent. At 100 TB of monthly egress, 97% offload means the origin ships 3 TB and 85% means it ships 15 TB - a five-fold difference in bandwidth, bill, and connection concurrency, from the same vendor and the same PoP map. Offload is a property of your cache keys and freshness headers, not of the network you bought.

Latency is equally mechanical. A cold HTTPS request spends one round trip on TCP and another on the TLS 1.3 handshake before a response byte moves. At 15 ms to a nearby PoP that is roughly 45 ms to first byte on a hit; against an origin 140 ms away it is over 400 ms, and every additional object pays the propagation delay again. Reliability is the axis teams find by accident: a cache with a sane stale policy keeps serving through an origin outage.

This article stays on the delivery path - for multi-region data placement and write routing behind the origin, see geo-distributed systems.

PoP topology and anycast catchment

A CDN announces the same IP prefix from every PoP over BGP. Which PoP a user reaches - that PoP's catchment - is decided by each intermediate network's best-path selection, so it tracks peering and transit economics rather than geography. A user in Lagos routinely lands in London because that is where their carrier peers with the CDN. Catchment is a routing artifact you observe, not a policy you set.

Inside a PoP the anycast address fronts many servers behind ECMP hashing on the 5-tuple, so every packet of a connection lands on the same machine - until the routing table changes. A BGP reconvergence or a PoP drain can move a live flow to a different site, where it is an unknown connection and gets reset. Short requests never notice; long downloads, WebSockets, and event streams are the workloads that report mysterious mid-transfer failures during CDN maintenance.

DNS steering is the alternative - different addresses per resolver, refined by EDNS Client Subnet. Finer control, but it inherits resolver caching and TTLs that recursives ignore. Anycast is stateless and divides a volumetric attack across every PoP that receives it.

The request path, hop by hop

Your zone publishes a CNAME to the CDN, which resolves to an anycast address. The client completes TCP or QUIC and a TLS handshake against the edge server; the CDN, not you, holds the connection.

The edge builds a cache key from the request and looks it up - memory first, then local SSD. A hit is written straight back. A miss does not go to your origin: it goes to a mid-tier shield over the CDN's own backbone, where the same lookup runs against a much larger cache holding the union of every nearby edge's misses.

Only a shield miss reaches the origin. If the shield holds a stale copy it revalidates rather than refetching, sending If-None-Match with the stored ETag; a 304 Not Modified refreshes freshness for a few hundred bytes instead of moving the whole object. The response is stored per its caching headers at both tiers, hop-by-hop headers stripped, before continuing to the client.

Every tier records what it did: Cache-Status (RFC 9211) appends one member per cache on the path, so a single curl shows which tier missed.

Cache keys are where hit ratio is won or lost

The default key is scheme, method, host, path, and the full query string. Everything that goes wrong with hit ratio is a key that varies for reasons the response bytes do not.

Query strings. One campaign appending utm_source, utm_campaign, and fbclid turns a single landing page into thousands of distinct objects, each fetched from origin once and never requested again. Normalize at the edge: sort parameters canonically and keep only an allowlist that genuinely selects content - page, sort, id - dropping the rest from the key while leaving the URL intact for the origin.

Vary. This response header adds named request headers to the key. Vary: Accept-Encoding is fine because normalization collapses it to two or three values. Vary: User-Agent is catastrophic - effectively unique per browser build, so the key space explodes and hit ratio approaches zero. Vary: Cookie is worse: it keys per session, and a response that escapes carrying a session cookie is a cross-user data leak, not merely a cold cache.

Normalize before the key is built: bucket the user agent into a device class, fold country into a currency enum, reduce cookies to the one boolean the page branches on, and vary on those derived values.

Freshness: three TTL layers and two stale directives

Browsers and CDNs read different fields. max-age applies to every cache including the browser; s-maxage overrides it for shared caches; Surrogate-Control and the newer CDN-Cache-Control are read by the CDN alone and stripped before the client sees them, so the CDN can hold an object far longer than any browser is permitted to.

A typical HTML policy is Cache-Control: max-age=0, s-maxage=600, stale-while-revalidate=86400. Browsers revalidate on every navigation, the CDN serves from cache for ten minutes, and for a day after that it serves the stale copy immediately while refreshing in the background. That last directive is the highest-leverage line in most configurations: nobody waits on a refill, so origin fetch latency leaves the user-visible tail entirely.

stale-if-error is its availability counterpart - keep serving expired content when the origin returns 5xx or times out. Inside that window an origin outage is invisible to readers of cached pages.

The failure mode to watch for is silence. A response with no Cache-Control and no Expires falls back to heuristic freshness, commonly 10% of the age since Last-Modified - a file untouched for a year gets cached for over a month. Always state a TTL, even if it is zero.

Invalidation versus versioned URLs

Purging works three ways: exact URL, path prefix, and surrogate key. Surrogate keys are the useful one - the origin tags a response Surrogate-Key: product-812 category-shoes, and a write to product 812 purges that tag, evicting every page carrying it without anyone maintaining a URL list.

Purge is fast but neither instant nor atomic: it fans out to every PoP over the control plane in a second or two, and in that window different users legitimately see different versions. The dangerous operation is purge everything - it empties the edges and the shield at once, so the next minute of ordinary traffic arrives at the origin entirely uncached, a self-inflicted thundering herd at exactly the moment you were trying to fix something. Treat it as an incident action, not a deploy step.

For anything the build produces, do not invalidate at all. Emit content-hashed filenames (app.7f3c9a1.js) served with max-age=31536000, immutable. New bytes get a new URL, old objects age out, and immutable stops browsers revalidating even on reload. Short-TTL HTML then points at long-lived assets, and the only thing that ever needs purging is the small mutable surface.

Origin shielding and collapsed forwarding

Without a shield every edge PoP is an independent cache. Two hundred PoPs each miss once per TTL, so one object with a 60-second TTL generates roughly 200 origin fetches a minute however well any single PoP performs. A shield collapses that into one fetch the edges reuse - which is why enabling shielding moves origin offload several points without touching a header.

Inside a tier the equivalent mechanism is collapsed forwarding: when N concurrent requests miss the same key, one goes upstream and the rest wait and share its response. nginx spells it proxy_cache_lock; Varnish does it by default. It is what stops a popular object's TTL expiry from firing thousands of simultaneous origin requests.

It has a sharp edge. If the response turns out to be uncacheable, the waiters cannot share it and are serialized behind that one fetch - a queue that turns a mildly slow endpoint into a p99 catastrophe. Varnish's hit-for-pass exists for exactly this: on seeing an uncacheable response it records a short-lived "do not collapse this key" marker so later requests pass through in parallel. Place the shield near the origin, not near users - its job is to minimize distinct origin connections. For the same herd one layer down, at the database shard, see hot-key mitigation.

TLS at the edge and accelerating what cannot be cached

The edge terminates TLS, selecting a certificate by SNI from a pool that may hold thousands of customer hostnames. Resumption tickets are usually per-PoP, so a mobile client whose catchment shifts between networks pays a full handshake again rather than resuming. TLS 1.3 0-RTT early data removes a round trip but is replayable by design: allow it only for idempotent requests.

The origin leg is a second TLS connection and needs its own authentication. If the origin accepts anonymous requests from the internet, an attacker who learns its address bypasses the CDN entirely - and with it your WAF, bot management, and rate limits. Use authenticated origin pulls with a CDN-issued client certificate, or at minimum a shared secret header plus an IP allowlist.

For genuinely uncacheable traffic - checkout, authenticated APIs, POSTs - nothing is cached and the whole benefit is transport. The edge holds warm, persistent, already-slow-started connections onward, so the request arrives on a pipe with a large congestion window instead of paying handshake round trips and slow start, over a backbone routed on measured latency rather than BGP hop count. The saving is handshake and ramp-up, not origin compute.

The metrics that actually diagnose a CDN

Request hit ratio and byte hit ratio, separately. They diverge whenever object sizes are uneven: 99% of requests can hit while a handful of missed video segments carry most of the bytes. Requests tell you about latency, bytes about the bill.

Hit ratio per tier. Split edge hit, shield hit, and origin fetch. A high edge ratio with a collapsing shield ratio means key cardinality is growing; a healthy shield masking a poor edge ratio means TTLs are too short for the per-PoP request rate.

Origin offload, one minus origin bytes divided by edge egress bytes. This is the dashboard number: it converts directly into money, and into how much origin capacity an incident will need.

p99 segmented by cache status. A blended percentile mixes a 20 ms hit distribution with a 400 ms miss distribution, so it moves whenever the ratio moves and tells you nothing about either. Hit p99 regressions are an edge problem; miss p99 regressions point at the origin or the backbone.

Cached object count against eviction rate. Object count climbing while hit ratio falls is the signature of a key explosion - a new tracking parameter, or a Vary someone added.

What a CDN will not fix

It fits static assets, media, software downloads, and read-heavy API responses that tolerate seconds of staleness - plus TLS termination, volumetric DDoS absorption, and availability through stale-if-error. Those are the cases where many users want identical bytes.

It does not fix per-user HTML. Personalized pages either bypass the cache or get restructured: cache a shared shell aggressively and fetch the personal fragment client-side, so the expensive part is cached for everyone. Write paths and strongly consistent reads pass through untouched, and no CDN makes a slow origin fast on a miss - it only ensures fewer requests become misses.

Do not let it become the only thing between users and an origin that cannot cope. Authentication, routing, and per-tenant policy belong at the API gateway, and the origin still needs its own load-shedding behaviour for the day a purge-all or a cache-busting bot turns the cache off.