Cruxly › System design › Design Exercises

Video Streaming

the bytes are too big to move at request time

Prompt: design a video streaming service — upload, process, and play back video to millions of concurrent viewers, adapting quality to each viewer's connection.

40 minutes first. Almost every decision follows from one fact: the data is enormous and the work has to happen before anyone presses play.

Applying the four moves: Scope — upload, transcode, adaptive playback to millions of concurrent viewers; recommendations and live streaming out. Estimate — egress bandwidth is the number, and it's large enough that it determines both architecture and cost; storage multiplies by the number of renditions you keep per video. Sketch — upload to object storage, an asynchronous transcoding pipeline producing chunked multi-bitrate renditions, and a CDN serving the segments. Deep dive — all the heavy work happens before anyone presses play, so the interesting failures are in the pipeline (a partial transcode, a stuck job) and in adaptive bitrate switching on the client, not in serving.

Scope and the numbers that force the architecture

In scope: upload, transcoding into multiple qualities, storage, adaptive-bitrate playback, CDN delivery. Out of scope: recommendations, comments, live streaming (genuinely different — no pre-processing window, so say it's a separate design), DRM specifics, monetisation.

Estimate. 500,000 concurrent viewers at an average 3 Mbps.

Egress: 5 x 10^5 x 3 x 10^6 bits/s = 1.5 Tbps sustained.
That number is the entire design. No origin infrastructure serves 1.5 Tbps economically. This must be served from a CDN, and that isn't an optimisation — it's the only viable shape. Saying this in the first three minutes is what separates a real answer from one that eventually rediscovers it.
Storage: 1,000 hours uploaded/day, and each source is transcoded into ~6 renditions totalling roughly 2-3x the source size. At ~2 GB/hour source, that's ~2 TB/day in, ~6 TB/day stored, growing forever. Object storage with lifecycle tiering, not a database.
Upload: modest in request count, enormous in bytes per request — a completely different traffic shape from playback, which is why they're separate paths with separate scaling.

What the numbers say. Read path: static, cacheable, edge-served, latency-critical. Write path: rare, huge, asynchronous, throughput-critical. These have almost nothing in common, and the design's first move is to stop treating them as one system.

Most numbers in a design interview inform a decision. This one IS the decision — there's no version of this system that doesn't route through a CDN.

Upload and the transcoding pipeline

Upload goes directly to object storage, not through your application servers. Issue a pre-signed URL and let the client PUT straight to storage. Routing gigabytes through your API tier wastes bandwidth twice and turns a slow upload into a held-open application connection. Support resumable/chunked upload, because a 2 GB upload over a mobile connection will be interrupted and restarting from zero is unacceptable.

Transcoding is the expensive part, and it is embarrassingly parallel. On upload completion, an event kicks off a pipeline:

@@ 1 | Split the source into chunks

Segment the source into independent pieces (a few seconds each) at keyframe boundaries. Because each chunk decodes independently, chunk N and chunk N+50 can be transcoded on different machines at the same time — this is what makes a 2-hour video's transcode take minutes rather than hours.

@@ 2 | Transcode each chunk into every rendition, in parallel

Each chunk is encoded into roughly six quality levels (240p through 4K), each at a different bitrate. This is a pure fan-out across a worker fleet: chunks x renditions independent jobs, each one CPU-bound, none depending on any other.

@@ 3 | Package into segments plus a manifest

Write the encoded segments (HLS or DASH format, typically 2-6 seconds each) and generate a manifest — a small text file listing every available rendition and every segment URL. The manifest is what the player reads; it is tiny, and it is the only thing that needs to be fetched before playback starts.

@@ 4 | Publish and warm the edge

Mark the video ready and, for content expected to be popular, push segments to edge caches before the first viewer arrives rather than waiting for a cache miss to pull them.

:::key Why chunked transcoding is the load-bearing decision It converts one enormous serial job into thousands of tiny independent ones. That means the pipeline scales by adding workers, a single machine failure loses seconds of work rather than hours, and a stuck job retries in isolation. Every desirable property of this pipeline comes from that one choice. :::

Every desirable property of the pipeline — parallelism, cheap retries, linear scaling — traces back to this one decomposition choice.

Adaptive bitrate: why the client decides

The player, not the server, chooses quality. It reads the manifest, starts conservatively, measures its own download throughput and buffer level, and steps up or down between renditions at segment boundaries.

This is worth stating as a deliberate design decision rather than an implementation detail, because it's what lets the server side stay completely stateless. If the server chose quality, it would need per-viewer connection state, real-time feedback, and an active session for every one of 500,000 concurrent viewers. Instead, every viewer is just fetching static files, and the CDN can serve them without knowing anything about who's watching or what came before.

Because renditions are pre-cut at identical segment boundaries, switching quality is just requesting the next segment from a different rendition. No session, no negotiation, no server-side state.

Segment length is a real trade-off worth naming. Short segments (2s) adapt to bandwidth changes quickly and start playback sooner, but produce more requests and slightly worse compression. Long segments (10s) compress better with fewer requests, but the player is stuck at a bad quality for up to 10 seconds after a bandwidth drop. Most services land at 2-6 seconds; the point is knowing why it's a dial rather than reciting a number.

Caching strategy and the failure modes

The catalogue is extremely skewed — a small fraction of videos serve the overwhelming majority of views. That skew is what makes CDN caching work at all, and it justifies a tiered approach: hot content pinned at the edge, warm content pulled on demand, cold content served from origin with a slower first segment.

Optimise the first segment specifically. Time-to-first-frame is the metric users actually feel, and it's dominated by whether segment 1 is at the edge. It's worth keeping just the opening segments of a large catalogue cached even when the rest of those videos aren't — the storage cost is small and it converts a slow start into a fast one across a much wider set of content.

Failure modes to raise unprompted:

Thundering herd on a new popular release. A million viewers request segment 1 of a video that isn't cached yet, and they all miss to origin simultaneously. Fixes: pre-warm before publishing for anything expected to be popular, and use request coalescing at the edge so one miss fetches from origin while the rest wait on that single fetch — the same single-flight pattern used against cache stampedes elsewhere in this track.

Transcoding fleet backlog. Uploads spike and the queue grows. Since transcoding is asynchronous by design, this degrades to "videos take longer to become available," which is the correct failure — but it needs an explicit priority policy, or a backlog of low-priority bulk uploads delays content people are waiting on.

A rendition fails to transcode. Publish the video with the renditions that succeeded rather than blocking the whole thing on one failed encode. A video available in 3 qualities beats a video available in none.

Partial upload / abandoned multipart. Incomplete uploads occupy storage indefinitely. A lifecycle rule to clean up incomplete multipart uploads after some days is unglamorous, real, and the kind of operational detail that signals having run something like this.

Regional imbalance. A video popular in one region shouldn't be replicated to every edge worldwide. Let the CDN's own pull-based caching handle geographic distribution naturally rather than pushing everything everywhere — push only for anticipated launches.

The same single-flight pattern that protects any cache from a stampede — here applied to the specific moment a video first goes viral.

How to say it in the interview

"The defining constraint is that a single video is gigabytes and there are millions of concurrent viewers, so nothing expensive can happen on the playback path. Everything — transcoding, segmenting, placement at the edge — happens ahead of time, and playback becomes static file delivery."

Checklist

Computed aggregate egress and used it to make CDN delivery non-optional
Split upload and playback into separate paths with different scaling properties
Uploaded directly to object storage via pre-signed URL, not through the API tier
Chunked the source at keyframe boundaries so transcoding parallelises
Named chunked transcoding as the decision every other pipeline property follows from
Put quality selection in the client so the serving side stays fully stateless
Explained segment length as a trade-off rather than quoting a number
Used catalogue skew to justify tiered caching, and optimised the first segment specifically
Handled the new-release thundering herd with pre-warming plus request coalescing
Published with partial renditions rather than blocking on one failed encode
Let CDN pull-based caching handle geography instead of pushing everywhere

Practice this interactively