Cruxly › System design › Design Exercises
File Storage and Sync
converging on the same bytes without re-sending what hasn't changed
Open the interactive lessonPrompt: design a file storage and sync service — like Dropbox. Users upload files, edit them on multiple devices, and every device should converge on the same content. A 2GB file with one changed paragraph should not require re-uploading 2GB.
40 minutes first. Storing a file is trivial. Efficiently syncing one that changed a little, on a device that might be offline for days, is the exercise.
Applying the four moves: Scope — upload, edit on multiple devices, converge, sync deltas not whole files; sharing permissions and real-time co-editing out (co-editing is the collaborative-docs exercise). Estimate — the 2GB file with one changed paragraph is the number that matters: re-uploading it is the cost the design exists to avoid. Sketch — files split into content-addressed chunks, a metadata service tracking which chunks make up each version, and dedup by chunk hash. Deep dive — a device offline for days comes back with edits that conflict with newer ones: name the conflict policy explicitly (keep both, labelled) rather than silently picking a winner.
Scope, and the number that changes the architecture
In scope: file upload/download, sync across multiple devices per user, detecting and propagating changes, basic conflict handling. Out of scope: real-time collaborative editing within one document (Google Docs' operational-transform problem is a genuinely different, harder problem — name it as such, don't attempt it), sharing/permissions specifics, versioning UI.
Estimate. Say 50 million users, average 5GB stored each, 3 devices per user syncing.
- Total storage: 50M x 5GB = 250 PB. Enormous, and almost entirely write-once-read-rarely — object storage with lifecycle tiering, the same answer as the video-streaming exercise elsewhere in this track.
- The number that actually drives the design is not total storage, it's the sync bandwidth for a small edit to a large file. A user editing one paragraph of a 500MB document and expecting near-instant sync across their other two devices cannot re-transfer 500MB — that single requirement is the entire reason this problem isn't "just S3 plus a database."
Chunking and content-addressing: the mechanism that makes small syncs cheap
Split every file into fixed- or variable-size chunks (a few hundred KB to a few MB each) and hash each chunk's content (SHA-256, typically). Store chunks keyed by their hash — content-addressed storage — and a file becomes a small manifest: an ordered list of chunk hashes that reconstruct it.
This is variable-size, content-defined chunking done properly — not fixed byte offsets. If chunk boundaries were fixed positions (every 1MB, say), inserting a single byte near the start of the file shifts every subsequent byte, changing every chunk's boundary and therefore every chunk's hash, even though almost nothing actually changed. Content-defined chunking (using a rolling hash to decide boundaries based on the data's own content, not a fixed offset) means an insertion only disturbs the chunk it occurred in — everything before and after realigns on the same boundaries it had before, because the boundary decision depends on local content, not position.
Deduplication falls out of this for free, across the entire user base, not just one user's file history. Two different users uploading the same stock photo, or the same paragraph of boilerplate license text, produce identical chunk hashes — the server already has that chunk and stores it once, referenced by both users' manifests. This is the same content-hash deduplication idea as the web crawler's content-seen check elsewhere in this track, applied to file bytes instead of page bodies.
Sync: detecting changes and propagating them efficiently
Each device maintains a local manifest and a "last known server state" marker. On reconnect (a device coming back online after being offline for days is the normal case here, not an edge case), the device asks the server "what changed since marker X" rather than re-scanning or re-hashing every file — the server tracks a monotonic change log per user, and the device requests the delta, the same "query by what's due" principle used throughout this track rather than a full re-scan.
On the sending side, only modified files get re-chunked and re-hashed. A filesystem watcher (or, cross-platform, periodic scanning with modified- timestamp comparison) identifies which files changed since the last sync; unmodified files are never touched, so a user with 200,000 files and one edited document only does chunking work on that one document.
Bandwidth-constrained sync (mobile, poor connectivity) needs its own policy, not just "sync everything." Defer large-file sync to Wi-Fi, prioritize recently-accessed files, and make partial/resumable chunk transfer the default — a chunk-level transfer that fails partway through only needs to resume that one chunk, not restart the whole file, which is exactly the granularity content-addressing already gives you almost for free.
Conflict resolution: two devices editing while offline
This is the actual hard problem in this exercise, not the storage mechanics. Device A and device B both had the file, both went offline, both made changes, and both come back online — the server now has two divergent versions descending from the same base.
Detecting the conflict requires versioning, not just timestamps. Give each file (or each chunk sequence) a version vector or a simple monotonically-increasing version number per device-file pair. When device B uploads its change, if the server's current version doesn't match what B last synced from, B was editing a version that is no longer the latest — that mismatch, not a timestamp comparison, is what reliably detects a conflict. Timestamps alone fail here because device clocks skew, exactly the clock-skew problem covered throughout this track's other exercises.
The honest options, each with a real cost:
- Last-write-wins. Simple, and silently discards one user's edits. Only acceptable when data loss is truly tolerable — genuinely rare for user files.
- Keep both, as a conflicted copy. Rename the losing version (
document (conflicted copy from device B).docx) and let the user reconcile manually. This is what most real sync services actually do, because it is the only option that never silently loses data — the cost is pushing the reconciliation work onto the user. - Automatic merge. Only tractable for structured, mergeable formats (a shared code file diffed line-by-line, a CRDT-backed document). For an opaque binary blob, there is no principled way to merge two divergent versions automatically — say this limit explicitly rather than implying a generic merge is always possible.
Naming that "keep both" is the pragmatic default for opaque files, and explaining why automatic merge doesn't generalize past structured formats, is the answer that shows real judgment — not picking one option and defending it as universally correct.
Failure modes and the honest edges
A chunk upload succeeds but the manifest update is lost. The chunk exists in storage, orphaned, referenced by nothing. Harmless but wasteful; a periodic garbage-collection pass removing chunks with zero manifest references reclaims the space — the same "referenced or reclaimed" logic as any content-addressed store.
Partial sync interrupted mid-file. Because sync operates at chunk granularity, an interruption loses at most the in-flight chunk, not the whole file — this is the direct payoff of chunking for reliability, not just for bandwidth.
A user deletes a file on one device while editing continues on another. Whether delete-wins or edit-wins is a product decision, not a purely technical one — flag it as a genuine ambiguity worth asking the interviewer about rather than picking silently, the same "notice the ambiguity, don't resolve it unprompted" instinct valued throughout this track.
Storage cost of keeping version history. Every version of every file retained forever is expensive at 250PB scale. A retention policy (keep every version for 30 days, then thin to daily, then weekly) trades history depth for storage cost — the exact same downsampling-ladder idea used for the metrics pipeline elsewhere in this track, applied to file versions instead of time-series points.
How to say it in the interview
"The number that matters is not total storage, it's how much of a changed file actually needs to move across the network — and the answer to that is chunking plus content-addressing, which also happens to solve deduplication for free."
Checklist
- Identified sync bandwidth for a small edit, not total storage, as the number driving the architecture
- Used content-addressed chunking so a file is a manifest of chunk hashes
- Chose content-defined chunking over fixed-offset chunking
- Got cross-user deduplication as a consequence of content-addressing, not a separate feature
- Had devices request a delta since a marker rather than re-scanning everything on reconnect
- Detected conflicts via version mismatch, not timestamp comparison
- Chose 'keep both as a conflicted copy' as the pragmatic default for opaque files
- Named the limit of automatic merge — only tractable for structured, mergeable formats
- Made sync resumable at chunk granularity so an interruption loses one chunk, not the whole file
- Flagged delete-versus-concurrent-edit as a product decision rather than resolving it silently