Cruxly

Cruxly › Cloud and DevOps › Cloud & DevOps: Containers and Orchestration

Resource Requests and Limits: Sizing a Pod That Survives Real Traffic

OOMKills, CPU throttling, and the noisy-neighbor problem

Open the interactive lesson

Prompt: you're deploying a service to Kubernetes for the first time. It ran fine in every local test. Set its resource requests and limits such that it survives a real traffic spike, doesn't get OOMKilled under normal load, and doesn't starve or get starved by other pods on the same node.

30 minutes first. The naive version copies the requests/limits from a tutorial or a similar-looking service. That number was sized for a different workload, and the gap only shows up under load, not in the demo.

Measure before you guess

The only reliable source for a resource limit is a load test against this specific service, not a number that "seems reasonable" or one copied from a similar-looking deployment. Run the service under realistic concurrent load (not a single sequential request) and observe actual peak memory and CPU via kubectl top pod or a metrics pipeline, then set limits with headroom above the observed peak — not headroom above a guess.

Memory limits need more headroom than CPU limits, because their failure modes are asymmetric. Exceeding a CPU limit throttles the process — it gets slower, but keeps running. Exceeding a memory limit kills the process outright. A CPU limit set 10% too low costs you latency; a memory limit set 10% too low costs you an OOMKill and a full restart, mid-request, for every user that pod was serving.

Language runtime matters more than most people expect. A JVM with a heap sized for the container's old memory limit will happily allocate up to that old heap size and get OOMKilled the moment the container's limit is lowered — the JVM doesn't automatically know the container shrank around it unless it's explicitly container-aware (-XX:+UseContainerSupport, on by default in modern JVMs, but worth verifying on anything long-lived). Go and Node.js have their own versions of this. Since Go 1.25, GOMAXPROCS follows the container's CPU limit automatically, but Go still does not set its memory limit from the cgroup — set GOMEMLIMIT to about 90% of the container's memory limit so the garbage collector works harder before the kernel's OOM killer gets involved.

A worked example, with the actual arithmetic a capacity-planning review would ask for. Say a load test against a Go service, driven at the production p99 request rate (not average — p99, because that's the traffic shape that actually determines whether the pod survives), shows a steady-state RSS of 340MiB, a peak working set of 360MiB, and CPU pinned at 0.6 cores at peak. The naive move is requests: {cpu: 600m, memory: 340Mi} — exactly what was observed, zero headroom. That's wrong for two independent reasons:

A defensible configuration, stating which observed number each value is headroom above:

That makes the pod Burstable. The exact percentages matter less than stating, in the pull request, which observed number each value is headroom above and why — so the next engineer revisiting this after an incident isn't reverse-engineering intent from a bare YAML file.

Requests set placement, limits set the kill switch

The scheduler only looks at requests when deciding where a pod fits. If a node has 2 CPU free and your pod requests 0.5 CPU, the scheduler considers that node a valid placement — regardless of the pod's limit. So a node can host pods whose limits add up to far more than it has. What happens next depends on the resource:

So set the two resources differently — this is the common 2025-26 position:

QoS classes, precisely. Guaranteed (every container's request equals its limit for both CPU and memory) gets the lowest OOM-kill priority and is required for exclusive CPU pinning with the CPU Manager's static policy — the reason to choose it for latency-critical work with whole-number CPUs. Burstable (requests below limits, or no CPU limit) is the usual choice for web services. BestEffort (no requests at all) is always over its requests, so it lands in the first group the kubelet evicts.

The exact mechanism, in numbers, because "it gets throttled" is not an actionable sentence without them. On cgroup v2, a Kubernetes CPU limit of "2" cores becomes cpu.max = 200000 100000: 200,000 microseconds of CPU time per 100,000-microsecond (100ms) period. (On cgroup v1 the same values lived in cfs_quota_us and cfs_period_us.) When the cgroup has used its 200ms of CPU time inside a period, every thread in it pauses until the next period starts. So 40 threads each running for 5ms at the start of a period use the full 200ms quota in the first 5ms of wall-clock time — and the whole container is then throttled for the remaining ~95ms, even though the node has idle cores the entire time. This is "micro-bursting." Averaged over a full second this looks fine; sliced into the 100ms windows the kernel actually enforces against, the process was fully saturated for a few milliseconds and hard-throttled for the rest.

Don't confuse micro-bursting with the famous kernel bug. Indeed's engineers (Dave Chiluk, 2019) found a different problem: CFS hands out quota to each CPU in small slices, and per-CPU slices left unused at the end of a period expired instead of being returned. On machines with many CPUs, that throttled multi-threaded apps that were using far less than their quota — their fix took one service's worst-case latency from over two seconds to about 30 milliseconds. It was fixed upstream in Linux 5.4 and backported, so it isn't what's throttling you on a current kernel. Micro-bursting, by contrast, is the controller working exactly as designed.

Garbage-collected runtimes are hit by this mechanism especially hard, for a specific, mechanistic reason. A GC pause is itself a burst of CPU work — if a collection cycle needs 90ms of CPU time against a 50ms quota, it consumes the first 50ms, gets throttled, and the pause itself now waits for the next period to begin before the remaining 40ms of GC work can run — meaning the CFS quota doesn't just slow the application, it directly extends the length of a GC pause the application is already blocked on, compounding exactly the kind of tail latency a CPU limit was supposedly protecting against.

This is not a monitoring gap fixable by scraping metrics more often — a 15-30s scrape interval (a common setting; Prometheus's built-in default is 1m) can't observe a throttling event that starts and ends inside one 100ms window. Use the cumulative counters instead, as a ratio: rate(container_cpu_cfs_throttled_periods_total[5m]) / rate(container_cpu_cfs_periods_total[5m]) — the share of periods in which the container was throttled. Field experience among platform teams treats roughly 20-25% throttled-periods ratio as the threshold worth investigating, and above roughly 50% as a load-bearing contributor to observed latency, not merely correlated with it.

Why raising the limit doesn't fully fix it, and the live industry debate this has produced. Raising the CPU limit raises the quota ceiling per period, which helps, but the underlying problem — a bursty, highly parallel process concentrating its work into a fraction of each period — persists at any limit unless the workload's own parallelism is bounded (capping a thread or goroutine pool so concurrent CPU demand within a single period stays under quota). This mechanism is well-known enough now that it has produced a genuine, current disagreement among practitioners: some platform teams advocate removing CPU limits entirely for latency-sensitive workloads (keeping only a CPU request for scheduling, accepting the noisy-neighbor risk in exchange for never throttling), reserving hard CPU limits for workloads where predictable, bounded resource usage matters more than tail latency. There is no universally correct answer here — it is a real trade-off between two failure modes (throttling-induced latency vs. noisy-neighbor risk), and the responsible position is knowing which one your specific workload can tolerate less, not picking a default and never revisiting it.

cgroup v2, and why the "which version" question mostly answers itself now. cgroup v1 split accounting across separate, sometimes inconsistent hierarchies; cgroup v2 unifies this into one hierarchy with cleaner, more consistent semantics — and as of recent Kubernetes releases (1.31 and later) and current base OS images (Ubuntu 22.04+, Amazon Linux 2023, Fedora 31+), cgroup v2 is the de facto default, not a toss-up requiring investigation on every new cluster. cgroup v1 has been in maintenance mode since Kubernetes 1.31, and recent kubelets refuse to start on it by default. What still bites is the reverse of what people expect: older runtimes that only understood v1 misread limits on v2 nodes. Check runtime versions against v2 support — for example JDK 8u372, 11.0.16 or 15+, Node.js 20.3+, and (before Go 1.25) automaxprocs 1.5.1+. The durable engineering takeaway predates any specific kernel version: a CPU limit is a claim about behavior under a real, bursty concurrency profile measured in 100ms slices, not a claim verified by a steady-state average that no real request pattern ever actually produces.

The runtime-awareness trap

A process that queries "total system memory" instead of its own cgroup limit will size itself for the wrong number, and this is a language/runtime detail that bites experienced engineers who assume it's already handled. Older JVMs, some Node.js configurations, and any language runtime that reads /proc/meminfo directly instead of the container's cgroup files will see the host's total memory, not the container's limit — and will happily allocate (or size a connection pool, or size an internal cache) based on that much larger number.

Verify container-awareness explicitly rather than assuming a modern runtime handles it. For the JVM specifically: confirm -XX:+UseContainerSupport is active (it has been on by default since JDK 10, but a base image or a flag override can disable it) and check -XX:MaxRAMPercentage is set relative to the container limit rather than an absolute heap size baked into a Dockerfile years ago for a different deployment target. Run java -XX:+PrintFlagsFinal -version | grep -i maxheapsize inside the actual container, not on a developer laptop, and confirm the printed value scales with the container's memory limit when you change it — if it doesn't move, the JVM is reading something other than the cgroup limit and every downstream calculation about headroom is built on a wrong number.

Which cgroup version is under you changes what "container-aware" even means. cgroup v1 split accounting across separate hierarchies (memory, cpu, cpuset each tracked independently, sometimes inconsistently across kernels); cgroup v2 unifies this into one hierarchy with more consistent, better-documented semantics, and is what the JVM's and most modern runtimes' container-detection code is actually validated against going forward. You can check a node with stat -fc %T /sys/fs/cgroup/ (cgroup2fs means v2; tmpfs means v1). The trap that actually caught teams ran the other way from what you might guess: many runtimes understood cgroup v1 first, so when nodes moved to v2, older runtime versions stopped finding the limit and fell back to host memory and CPU counts. Upgrade to versions with v2 support (JDK 8u372 / 11.0.16 / 15+, Node.js 20.3+, Go 1.25+ or automaxprocs 1.5.1+) rather than assuming "container-aware" covers both.

This failure is invisible locally and immediate in production, because a local Docker run on a developer's laptop often has no memory limit set at all, so the runtime sees the laptop's full RAM and everything looks fine — right up until the exact same image runs inside a Kubernetes pod with a real, enforced limit. The fix that actually catches this before production: run the container LOCALLY with an explicit, tight memory limit that matches what production will set (docker run -m 480m ..., not an unbounded docker run), specifically so this class of bug fails in a laptop terminal instead of a pager alert.

How to say it in the interview

"Copying resource limits from another service's YAML is copying someone else's load profile, not yours. The only trustworthy number comes from actually watching this specific service under a realistic load test — CPU and memory both spike differently depending on what the code actually does, not what it looks like it should do."

Checklist

Practice this interactively
Share this lessonWhatsAppLinkedInX

Related lessons