Cruxly › System design › Principles of Distributed Design

State Gravity

state is the expensive thing; keep the set of things that have it small

The principle: every piece of state you add is a thing that must be replicated, backed up, migrated, made consistent, and recovered after a failure. Stateless components have none of those obligations. Therefore: keep the set of stateful components as small as the problem allows, and know exactly which ones they are.

The name is the point — state has gravity. It pulls complexity toward it: a stateless service can be killed and restarted with no ceremony, while a stateful one drags a recovery story, a replication story, and a migration story behind it forever.

The violation: state smeared across every service

The common version of this mistake does not look like a mistake. It looks like convenience.

API service        → holds user sessions in local memory
Recommendation svc → caches computed feeds in a local map
Upload service     → keeps partial uploads on local disk
Search service     → holds a locally-built index

Each decision was individually reasonable — local is fast, and nobody wanted a network hop.
But the system just acquired four independent recovery problems, and none of them are visible until something restarts.

The cost lands on the day you scale or a machine dies:

Kill the API service instance and users are logged out, because their session lived in that process.
Add a second recommendation instance and users get different feeds depending on which one they hit.
Deploy the upload service and in-flight uploads vanish.
The search index must be rebuilt per machine, so every new instance is useless until it warms.

The deeper cost: you can no longer scale the halves independently. Traffic doubling is now a coordination problem — you cannot simply add stateless machines, because there are no purely stateless machines left.

The fix: push state into a small number of components that specialise in it

API service        → stateless; session token validated against a shared store
Recommendation svc → stateless; reads/writes a shared cache
Upload service     → stateless; parts go straight to object storage
Search service     → stateless query layer over a shared index cluster

Stateful set: [ session store, shared cache, object storage, index cluster ]

The services are now interchangeable. Any instance can serve any request, a dying machine takes nothing with it, and scaling the request tier is adding machines with no coordination at all.
Notice what actually happened: the state did not disappear. It moved into four components whose entire purpose is holding state correctly — they have replication, failover and durability built in, maintained by people who specialise in exactly that. You did not remove the difficulty; you concentrated it where it is already solved.
This is why "stateless services" is not the principle. The principle is that the stateful set should be small, explicit, and made of components chosen for that job. A design with zero stateful components is not achieving anything except storing nothing.

:::key The question this principle makes you ask Not "is this service stateless?" but "can I list, out loud, every stateful component in this design?" If the list is long, or if you have to think hard to produce it, state has smeared — and the recovery story is already more complicated than you believe it is. :::

What the principle costs, honestly

A network hop on the hot path. Reading a session from a shared store is slower than reading it from local memory — always. If the latency budget genuinely cannot absorb that hop, keeping state local is the correct decision, and the right move is to say so explicitly: "I'm keeping this in local memory despite the recovery cost, because the budget is 20ms and a round trip spends a quarter of it."
A new dependency that can be down. The shared store is now on the critical path for every request. This trades "one machine's failure loses its own users' sessions" for "the store's failure loses everyone's" — a smaller blast radius per incident, a larger one when it happens. That is usually the better trade, because the shared store is built for availability in a way your API instance is not, but it is a trade and should be named as one.
Operational cost. Four stateful components is four things to monitor, patch and capacity-plan. A small team running a modest system may legitimately choose fewer moving parts over cleaner scaling.

:::key The violation worth making out loud Local caches that are pure performance optimisation — where a cold instance is slower but never wrong — do not violate this principle, because they hold no truth. The test is not "is there data in memory" but "if this process dies, did the system lose a fact it cannot reconstruct?" If the answer is no, it isn't state in the sense that matters. :::

How to say it in the interview

"Before I scale anything, I want to separate the parts of this system that hold state from the parts that do not, because those two halves have completely different scaling and failure stories — the stateless half I can add machines to freely, and the stateful half is where all the real difficulty is going to live."

Checklist

Listed every stateful component in the design out loud, as a set
Separated the stateless tier from the stateful set before discussing scaling
Said explicitly that state was relocated into specialised components, not eliminated
Named the latency cost of moving state out of process, rather than presenting it as free
Distinguished a pure performance cache from real state by asking if the fact is reconstructible
Was willing to violate the principle deliberately when the latency budget demanded it, and said so

Practice this interactively