Cruxly › System design › Scaling Building Blocks

Content Delivery Network

caching close to the reader, invalidating across nodes you don't control individually

Prompt: design a CDN — a network of edge servers, geographically distributed, that caches and serves static content (images, video segments, JS/CSS bundles) close to end users, with an invalidation mechanism for when origin content changes.

40 minutes first. The caching itself is easy. The interesting question is what happens across hundreds of edge locations when the origin changes.

Applying the four moves: Scope — geographically distributed edge caches serving static content, plus invalidation when the origin changes; dynamic content and edge compute out. Estimate — number of edge locations and hit ratio, since the hit ratio is what decides how much load still reaches the origin. Sketch — request routing to the nearest edge (DNS or anycast), edge caches, and origin shielding through a mid-tier so the origin isn't hit once per edge. Deep dive — the prompt's real question: invalidating across hundreds of edges is slow and partial, which is why versioned URLs (a new filename per change) beat purge calls for anything that changes deliberately.

Scope, and the routing problem before the caching problem

In scope: edge caching of static assets, request routing to the nearest healthy edge, cache invalidation/purging, origin shielding. Out of scope: dynamic/personalized content acceleration (a much harder, different problem), the actual video-encoding pipeline (covered in this track's video-streaming exercise, which this design sits downstream of).

The first design decision is how a user's request finds the nearest edge node at all — this is a DNS and/or Anycast routing problem, not a caching problem, and it comes before caching even enters the picture. Two real approaches: DNS-based routing resolves a CDN hostname to a different edge IP depending on the resolver's geographic location (GeoDNS); Anycast advertises the same IP address from every edge location and relies on internet routing (BGP) to naturally deliver a packet to the topologically nearest one that's currently announcing it. Anycast has the advantage of near-instant failover (if an edge node stops announcing the route, traffic automatically reroutes to the next-nearest one with no DNS TTL delay to wait out); GeoDNS is simpler to reason about and debug but inherits DNS's caching/TTL delay when routing decisions need to change quickly.

Cache hierarchy and origin shielding

The shield layer is what keeps a simultaneous cold-cache event across hundreds of edge nodes from becoming a thundering-herd origin overload.

A two-tier cache hierarchy — edge nodes plus a smaller number of regional "shield" caches sitting between edges and the origin — solves a real problem: without it, a cache-cold popular asset causes every one of hundreds of edge nodes to independently request it from the origin at roughly the same time, the classic thundering-herd problem, here at CDN scale. Origin shielding routes edge cache misses to a shield layer first, which itself caches; only a shield-layer miss reaches the origin, collapsing potentially hundreds of simultaneous origin requests into effectively one per shielded region. This is the same "single-flight" request-collapsing idea used in this track's caching material, applied at a larger geographic granularity.

Invalidation across a fleet you don't control individually

When origin content changes, every edge node holding a stale cached copy needs to find out — and there is no single "flush this one cache" call that works, because there are hundreds of independent caches. Two real approaches, with a real tradeoff:

TTL-based expiry. Every cached object carries an expiration time; an edge node simply refuses to serve it past that time and re-fetches from origin/shield. Simple, and creates a bounded staleness window (content can be wrong for up to the TTL duration) that must be set deliberately — a 24-hour TTL for a company logo is fine, a 24-hour TTL for a price that just changed is not.
Active purge/invalidation. The origin pushes an explicit "this URL is now stale" message to every edge node. This gives immediate consistency but at real operational cost: a purge request has to reliably reach every one of potentially hundreds of edge nodes, and a purge that fails to reach even one node leaves it silently serving stale content indefinitely with no TTL to eventually correct it — active purge trades a bounded staleness window for the risk of an unbounded one if delivery isn't reliable.

Versioned URLs sidestep the whole invalidation problem for build-artifact-style assets (app.a3f92c.js instead of app.js) — a new deploy produces a new URL, old cached copies simply become unreferenced rather than needing to be invalidated at all, and this is why static-asset pipelines almost universally do this for JS/CSS bundles. It doesn't help for content whose identity is meant to stay stable at one URL (a product image at a permanent link, an article page) — naming which category a given asset falls into, and picking the corresponding strategy, is the actual decision this section is testing.

Failure modes and the honest edges

An edge node goes down. Traffic reroutes to the next-nearest healthy edge — trivial with Anycast (routing adjusts automatically), requires active health-checking and DNS updates with GeoDNS, which is slower to converge.

Cache stampede on a very popular, newly-invalidated object — even with shielding, if TTL expiry for a viral asset happens to land at the same moment across many edges, the shield layer itself can face a burst. Request coalescing at the shield (holding concurrent requests for the same key and serving all of them from one origin fetch, rather than issuing N origin fetches for N simultaneous requesters) closes this remaining gap.

Purge propagation delay is often glossed over as instant — it isn't. An active-purge design still has non-zero propagation time across a global network of edge nodes; briefly, different users in different regions can see different content post-update. Stating this window explicitly, rather than implying purge is instantaneous everywhere, is the honest answer.

Range requests for large files (video seeking, resumable downloads) need edges to cache and serve byte-range subsets correctly, not just whole-file objects — a naive cache implementation that only knows how to store and serve complete responses breaks video seeking entirely, which connects directly to this track's video-streaming exercise's chunking approach.

How to say it in the interview

"Caching one thing close to one user is trivial; the actual design problem is that there are hundreds of edge locations that each independently decided to cache a copy, and none of them find out the origin changed until something tells them."

Checklist

Solved request routing to the nearest edge before addressing caching itself
Compared Anycast and GeoDNS on failover speed specifically, not just 'both route to the nearest edge'
Introduced a regional shield layer to prevent a cache-cold popular asset from causing a simultaneous origin thundering herd
Named origin shielding as the same single-flight request-collapsing idea used in general caching, at larger geographic granularity
Chose between TTL expiry and active purge with an explicit staleness-window tradeoff, not by default
Used versioned URLs for build-artifact assets to sidestep invalidation entirely, and named which assets that doesn't help
Added request coalescing at the shield layer for viral-object stampedes that TTL expiry alone doesn't prevent
Stated purge propagation delay explicitly rather than implying active invalidation is instantaneous everywhere
Required edge caches to correctly serve byte-range requests, not just whole-file objects

Practice this interactively