Fundamentals

Capacity & Latency Basics

Estimate QPS, storage, and latency budgets with back-of-the-envelope numbers.

Beginner, about 45 minutes to read, 3 open questions to practise.

1Understand

The idea

Systems interviews start with scale, not with boxes. Before you invent services. You and the interviewer need a shared picture of how big the system is: how many requests hit it, how much data it stores, and how fast a request is allowed to be.

Capacity estimation is that conversation. You turn a product story (“ten million users, each opens the feed a few times a day”) into QPS, storage growth, and a latency budget, with every assumption said out loud. The numbers do not have to be perfect; they have to be *defensible* and consistent, so later choices (cache, shard, queue) have something to hang on.

Strong candidates also sanity-check read/write ratio, payload size, and fan-out (one write → how many reads or notifications). Those numbers decide whether Redis, replicas, or queues are mandatory, not optional polish. Treat capacity math as a collaborative whiteboard: state assumptions, invite tweaks, then derive implications (“at ~2k peak QPS on this endpoint, uncached Postgres is the bottleneck”).

Intuition

Think in rates and sizes, not vibes.

Requests. If you know daily active users (DAU) and how many times each user hits a hot endpoint, average QPS is:

QPS_avg ≈ DAU × actions_per_user_per_day / 86_400

Peak is not the average. Product traffic has morning spikes, launches, and celebrity moments. A common interview move is QPS_peak ≈ QPS_avg × 3…5 unless the prompt gives a better factor. Size the *peak* for CPU and connection counts; use average for long-term cost talk.

Storage. Separate from QPS. Estimate *records written per day × bytes per record × retention*. A chat app’s message table grows with write rate and retention; a cache does not replace that durable store.

Latency. Name a budget (e.g. “p99 under 200ms for the read API”). Disk seeks, cross-region RTT, and cold DB queries eat that budget fast; in-memory and same-region caches are how you stay inside it. You do not need exact hardware numbers, you need to show that *disk vs memory vs network* are different orders of magnitude in your head.

The skill is not calculator speed. It is refusing to design until the scale is on the whiteboard.

Bandwidth back-of-envelope. If peak GET /feed is 740 QPS and each JSON is ~40 KB, egress is roughly 740 × 40 KB ≈ 30 MB/s (~240 Mbps) before replicas and retries, enough to mention CDN or edge cache for static slices of the feed, not just “add Redis.”

Read vs write. If users post 1×/day but open the feed 20×/day, the system is overwhelmingly read-heavy, cache and read replicas earn their keep; sharding the write path can wait. If writes are 40% of traffic with heavy indexing. Say that write QPS and index growth drive DB choice first.

Retention multipliers. Storage estimates need replicas (2-3×), indexes (often 0.5-2× row size for hot indexes), and backups (cold storage cost). Say “~3 TB logical, ~6 TB with replicas and indexes” so you do not under-quote disk for ops.

Interview ritual. Clarify MAU vs DAU, peak hour vs daily average, and geo, then compute. Pause after QPS: “That implies we need a cache layer or read replicas before micro-sharding.” Numbers are hooks for architecture, not trivia.

What you will be able to do

  • State explicit assumptions before calculating
  • Convert DAU and actions into average and peak QPS
  • Separate storage growth from request rate

When this comes up

  • How many requests per second?
  • How much storage in a year?
  • What is the p99 latency budget?

2See it

Architecture sketch

What the numbers buy you

Capacity estimates decide whether one box is enough, before you draw services.

100%
ClientServiceStore

Walkthrough

Numbers decide the next box

From peak QPS to “do we need a cache?”

1 / 4

100%

You estimated ~740 peak QPS for GET /feed. Write that on the board before drawing services.

Read the walkthrough as text

Worked example, news feed opens.

Prompt: 2M DAU; each user opens the home feed ~8 times/day; we will design the GET /feed path.

  1. Assumptions (say them): 2M DAU, 8 feed opens/user/day, read-heavy, peak ≈ 4× average, payload later.
  2. Average QPS: 2e6 × 8 / 86400 ≈ 185 QPS.
  3. Peak QPS: 185 × 4 ≈ 740 QPS. That is the number for “can one DB handle this?” and “do we need a cache?”.
  4. Storage (separate): if each open also logs an analytics event of ~200 bytes, that is 2e6 × 8 × 200 ≈ 3.2 GB/day of logs, different problem from serving the feed HTML/JSON.
  5. Latency budget: “p99 under 200ms for feed JSON.” Crossing regions or hitting disk on every request will blow that; a hot cache or precomputed feed for heavy users becomes motivated *by these numbers*, not by fashion.

What interviewers listen for. You stated assumptions before arithmetic. You did not treat average as peak. You did not mix “per day” into a per-second capacity without dividing by 86 400. You separated *traffic* from *bytes on disk*.

  1. Write path sketch: 5% of feed opens trigger a “refresh recommendations” job, 740 × 0.05 ≈ 37 background jobs/s; queue or worker pool, not synchronous in GET /feed.
  2. Follow-up they may ask: “What if launch day is 10×?”, you already have a peak factor; discuss provisional cache, pre-warm, or rate limit at edge.
  3. Phrase to use: “I sized peak QPS for connection pools and CPU; average QPS for monthly cost and storage growth.”

Push notifications variant (quick): 2M DAU, 20% enable push, 2 pushes/user/day → 2e6 × 0.2 × 2 / 86400 ≈ 93 enqueue ops/s to notification workers, separate from feed QPS; shows you do not lump all “events” into one number.

Napkin math

Napkin math, feed opens

DAU = 2,000,000

actions/user/day = 8 feed opens

requests/day = 2e6 × 8 = 16e6

QPS_avg ≈ 16e6 / 86_400 ≈ 185

peak factor = 4× → QPS_peak ≈ 740

p99 budget (say): < 200ms for GET /feed

Say assumptions before arithmetic. Size for peak, not average.

3Answer it

How to answer in an interview

Use this outline when a prompt asks “how big is this?” or before any box diagram:

  1. Clarify, DAU vs MAU, which endpoint, read/write mix, geo.
  2. Assumptions. Write them; invite the interviewer to adjust.
  3. QPS, average, then peak factor.
  4. Storage, growth rate and retention if durable data matters.
  5. Latency / availability, one sentence on the budget the design must hit.
  6. Implication, “at ~Xk peak QPS, a single primary will struggle on uncached reads” → leads into caching or sharding.

Keep units in the sentence: “~740 requests per second at peak,” not “about 700.”

Phrases that land: “Let me state assumptions first.” · “Peak is about 4× average unless you have a better launch profile.” · “That’s ~X GB/day durable; replicas and indexes often double footprint.” · “This read-heavy ratio motivates cache before shard.” · “If payload is Y KB, egress at peak is on the order of Z MB/s.”

  1. Sanity-check one constraint, p99 latency, 99.9% availability, or max acceptable staleness, so later patterns (cache TTL, replica lag) connect back to this napkin math.

Answer checklist

1. State assumptions (DAU, actions/user, read/write mix). 2. Derive QPS and peak (×3-5). 3. Estimate storage growth. 4. Name the latency budget and what must stay in-memory vs disk.

Tradeoffs & numbers

Rough orders of magnitude you can cite calmly (exact chips vary; direction matters):

  • Memory reference ≪ same-datacenter RTT ≪ cross-region RTT ≪ disk / cold DB.
  • A single well-indexed SQL primary often handles on the order of thousands of simple QPS before you need read replicas or caches, but a heavy join per request does not.
  • Over-estimating by 10× is safer than under-estimating by 10× when choosing whether a cache is mandatory; under-estimating peak is how designs die in launch week.
  • Push back on fake precision: three significant digits on a napkin estimate look like cargo-cult. One or two is enough.

When NOT to over-index on capacity. Tiny internal tools or early MVP with <100 users, one sentence of scale then design for 10× growth without full spreadsheet theater. If the prompt is purely “design URL shortener,” still derive click QPS before pick base62 vs snowflake.

  • API vs batch: nightly ETL of 500M rows is “few thousand rows/s for hours”, different from synchronous user QPS; do not divide batch totals by 86 400 as if uniform.
  • Cost vs safety: over-provisioning caches at day one burns money; under-provisioning on peak burns credibility in the interview.
  • Celebrity / flash crowd: peak factor 4× may be too low for ticket sales or live streams, say you would load-test or use historical launch data.

Kinds of questions (3)

Same pattern, different interview surfaces. Recognise which variant you are in before you answer.

  • QPS from DAU

    Example. 5M DAU, 3 searches/user/day, what is average and peak QPS for search? On the whiteboard, label units (per-second vs per-day), state peak ≈ 3-5× average unless the prompt gives a launch spike, and tie the number to what you provision next.

    How. QPS_avg = 5e6 × 3 / 86400; multiply by an agreed peak factor (e.g. 4×). State both numbers. Sanity-check arithmetic aloud; invite the interviewer to change DAU or actions/day; end with one provisioning implication.

  • Storage over time

    Example. 2M messages/day, 500 bytes each, keep 2 years, how much disk? Roll up `messages/day × bytes/row × retention days`, then add replicas, secondary indexes, and backup snapshots (often 2-3× raw data).

    How. Daily bytes × 365 × 2; say whether replicas/indexes multiply storage (often 2-3×). Call out growth rate vs one-time size; note when you shard or tier cold data to object storage.

  • Latency budget vs disk seek

    Example. p99 must be under 100ms for a profile read. Draw API → cache → DB and allocate milliseconds per hop; a single uncached disk seek on the hot path usually violates the budget at scale.

    How. Argue that a cold disk-bound query per request is risky; prefer memory/cache or precompute for the hot path. Compare memory (~µs), same-AZ RTT (~ms), and disk seek (~ms) when justifying cache or read replicas.

Common mistakes

Failure modes the AI coach watches for when you practise open questions.

Average vs peak

Avoid. Size only for average QPS. Launch day melts the primary.

Prefer. State peak ≈ average × factor (or measured peak) and design for that.

Weak sketch

Bad

100%

Stronger sketch

Good

100%
  • No assumptions

    Why it happens. Jumping to “so about a million QPS” sounds confident but cannot be checked or adjusted.

    How it fails. “We will get huge traffic so we need Kafka and 20 services” with no DAU, actions, or peak factor.

    Fix. Write 3-5 assumptions first; derive QPS from them; invite the interviewer to change an assumption.

  • Average as peak

    Why it happens. Hardware and connection pools die at the spike, not at the daily average.

    How it fails. Sizing a single DB for 200 QPS average when peak is 800-1000.

    Fix. Always state peak = average × factor (or a measured peak). Design caches/replicas for peak reads.

  • Mixed units

    Why it happens. Per-day and per-second differ by ~10⁵; mixing them invents fantasy scale.

    How it fails. Treating “16 million requests/day” as “16 million QPS”.

    Fix. Divide day counts by 86 400 for QPS. Label every number with its unit.

4Practise

Concept check

A short quiz on this lesson before open questions. Optional, and it does not gate practice.

Open questions

Ready to practise?

Write an answer, get AI coaching, then revise. You are not building a real system.

Practise with an open question

All systems lessons