Caching & Scale

Caching

Put a cache in the read path: what to store, where, and how it goes stale.

Beginner, about 60 minutes to read, 4 open questions to practise.

1Understand

The idea

A cache remembers the result of an expensive read so most traffic never pays that cost again. In interviews, “add Redis” is not the answer. The answer is *where* the cache sits on the path, *what* the key and value are, *how* writes make entries wrong, and *what* happens when many clients miss together.

Caching is a deliberate lie: you serve data that might be slightly old in exchange for latency and origin load. Good designs make that lie explicit (TTL, invalidation, consistency story). Bad designs cache the wrong layer, the wrong key, or forever.

Interviewers want you to quantify hit-rate intent (“~90% of reads hit the top 1k SKUs”) and to spell failure mode: Redis down → cap DB concurrency and degrade, not 500 the fleet. Caching trades freshness and memory for latency. Say that trade explicitly.

Intuition

A cache pays off when the same expensive read repeats. The expensive part might be a DB query, a join, a remote service, or a rendered blob. Caching random writes or one-off personalized payloads with a shared key usually hurts.

Where it sits decides what it can help:

  • CDN / edge, public, cacheable responses (images, static JS, public product pages). Great for locality; wrong for per-user secrets.
  • Application / Redis, authenticated API reads, product JSON by id, session-ish data with an explicit key.
  • In-process, tiny ultra-hot keys inside one instance; does not help other instances and dies on deploy.

Keys must encode everything that changes the answer. product:42 is fine for a public product document. A “recommended for you” feed needs rec:{userId} (or similar). Missing the user id from a personalized key is a privacy bug, not a style issue.

Freshness is the hard part. Options: TTL only (simple, stale until expiry), invalidate-on-write (delete or overwrite the key when the source changes), write-through (update cache on write). Real systems often combine: invalidate on known writes + TTL as a safety net.

Stampede (thundering herd): a hot key expires and thousands of clients miss together, all hitting the origin. Mitigations: singleflight / request coalescing, soft TTL with background refresh, probabilistic early expiration,

Negative cache, short TTL tombstone for known-missing keys so repeat misses do not DDOS origin. Layers: browser → CDN (public) → Redis (auth reads) → DB; each layer has different cache key rules.

Cache-aside default: app reads Redis, on miss loads DB and sets key; on write, commit DB then invalidate, never assume TTL alone for admin-visible fields.

Cache-aside default: app reads Redis, on miss loads DB and sets key; on write, commit DB then invalidate, never assume TTL alone for admin-visible fields.

What you will be able to do

  • Place a cache for a concrete read path
  • Define keys and invalidation
  • Call out stampede and consistency tradeoffs

When this comes up

  • Hot keys
  • Read-heavy traffic
  • CDN vs app cache
  • Cache invalidation

2See it

Architecture sketch

Read-through cache

Cache sits in front of the expensive read. Postgres stays source of truth.

100%
ClientServiceCacheStore

Walkthrough

Request path

Hit vs miss on GET /products/{id}

1 / 5

100%

Client calls GET /products/42. API owns the read path.

Read the walkthrough as text

Case, product detail API.

GET /products/{id} is 90% of traffic, hits Postgres every time, p99 120ms, DB CPU hot. Admins occasionally PATCH the title.

Read path (read-through).

  1. Client → API.
  2. API looks up Redis key product:{id}.
  3. Hit → return JSON (fast path).
  4. Miss → query Postgres → store JSON in Redis with TTL (e.g. 5-15 min) → return.

Write path.

  1. Admin updates title in Postgres (source of truth).
  2. Invalidate product:{id} (or set the new JSON). Prefer explicit invalidation over hoping a long TTL expires.
  3. Next reader refills from Postgres.

Why not CDN for this JSON? If the API is authenticated or the document is not publicly cacheable, a shared edge cache is the wrong layer. CDN *images* for the product gallery; Redis for the API document.

Stampede on a celebrity product. When product:celeb expires, do not let 5k misses hit Postgres. Coalesce: one request fills Redis while others wait; or refresh before hard expiry.

What a strong answer names. Placement (Redis in front of Postgres), key shape, invalidate-on-write + TTL backstop, stampede, and the consistency story (“readers may see old title for a moment

Listen for: hit vs miss latency, hit-rate metric, stampede on viral SKU, no global key for personalized JSON. Optional stale-if-error if origin unhealthy.

  1. Deploy safety: bump key namespace product:v2:{id} when response schema changes during rollout.

What a strong answer names. Placement (Redis in front of Postgres), key shape, invalidate-on-write + TTL backstop, stampede, and the consistency story (“readers may see old title for a moment during a race”).

Napkin math

Cache napkin

key = product:{id}

hit ratio target ≥ 90% on hot path

TTL = 5-15 min (or invalidate on write)

miss → DB → fill (read-through)

stampede: singleflight / lock / jittered TTL

uncached QPS × (1 − hit) ≤ DB budget

Name the key and who invalidates it. Cache is not the source of truth.

3Answer it

How to answer in an interview

When the prompt hints at hot reads or slow DB:

  1. Name the expensive read you are protecting.
  2. Place the cache on that path (and say what you are *not* caching).
  3. Key + value, what uniqueness dimensions?
  4. Invalidation. Write path, TTL, or both.
  5. Failure modes, stampede, stale reads, cache down (fallback to DB?).
  6. Tradeoff, freshness vs load; why this layer and not CDN/in-process.

Avoid “we will cache everything in Redis” with

Say: “Key includes every dimension that changes the answer.” · “Write path: SQL first, then invalidate.” · “Hot key: singleflight.” · “Miss storm: monitor origin QPS spike.”no keys and no write story.

Answer checklist

1. Name the expensive read you want to avoid. 2. Choose cache location (CDN, edge, app, Redis) and key shape. 3. TTL vs explicit invalidation. 4. Say what happens on miss, stampede, and stale data.

Tradeoffs & numbers

  • Hit rate matters more than micro-optimising miss latency; wrong keys → low hit rate.
  • Invalidate-on-write needs a reliable signal from every writer; miss a writer and you serve ghosts.
  • TTL-only is simpler ops, worse freshness, fine for soft data, bad for admin-edited titles users notice.
  • Cache down: design should degrade to origin (maybe with shed load), not 500 the fleet.
  • Memory cost: giant values and high cardinality keys (rec:{user} for every user) can make Redis the new bottleneck, say when you would paginate, compress, or not cache.
  • When NOT to cache: no reuse, strict audit reads, or per-request unique payloads.
  • Stampede math: 5k parallel misses on one key without coalescing can saturate origin in seconds.
  • CDN vs Redis: shared URL only for identical bytes across users.

Kinds of questions (4)

Same pattern, different interview surfaces. Recognise which variant you are in before you answer.

  • Read-through app cache

    Example. Product JSON API hammering Postgres. Say cache key shape, expected hit rate on the hot path, and what happens to Postgres p99 when Redis fails or a hot key expires together.

    How. Redis in front; miss loads DB and fills cache; key = product id. Draw read path with hit/miss; on write describe invalidate vs update; mention stampede guard on celebrity keys.

  • CDN for static / public content

    Example. Product images and public marketing pages. Say cache key shape, expected hit rate on the hot path, and what happens to Postgres p99 when Redis fails or a hot key expires together.

    How. Edge cache with cache-friendly URLs; do not put personalized JSON on a shared CDN URL. Draw read path with hit/miss; on write describe invalidate vs update; mention stampede guard on celebrity keys.

  • Write-through vs invalidate-on-write

    Example. Admin edits a title; users must see it quickly. Say cache key shape, expected hit rate on the hot path, and what happens to Postgres p99 when Redis fails or a hot key expires together.

    How. Prefer invalidate or update the key on write; TTL alone can leave stale titles for minutes. Draw read path with hit/miss; on write describe invalidate vs update; mention stampede guard on celebrity keys.

  • Hot-key protection

    Example. Celebrity id expires and thunders the DB. Say cache key shape, expected hit rate on the hot path, and what happens to Postgres p99 when Redis fails or a hot key expires together.

    How. Singleflight / soft TTL / early refresh so one miss fills for many waiters. Draw read path with hit/miss; on write describe invalidate vs update; mention stampede guard on celebrity keys.

Common mistakes

Failure modes the AI coach watches for when you practise open questions.

Wrong layer vs right layer

Avoid. Personalized feed on a shared CDN URL

Prefer. Public images on CDN; feed in Redis with user key

Weak sketch

Bad

100%

Stronger sketch

Good

100%
  • Cache everything

    Why it happens. Caching unique, write-heavy, or wrongly keyed data wastes memory and serves junk.

    How it fails. Caching each personalized feed at `feed:global` so every user sees someone else’s recommendations.

    Fix. Cache repeated expensive reads; put user (or session) in personalized keys; skip one-off writes.

  • No invalidation

    Why it happens. TTL-only means admins and users disagree about reality until expiry.

    How it fails. Title updated in DB; Redis still serves the old title for 30 minutes with no invalidate.

    Fix. On write, delete or overwrite the key; keep TTL as a backstop, not the only freshness mechanism.

  • Wrong layer

    Why it happens. A cache below the bottleneck or at the wrong trust boundary does not help.

    How it fails. Putting a CDN in front of an authenticated per-user JSON API, or caching after the slow join already ran.

    Fix. Place the cache immediately in front of the expensive step; use CDN for public/static, app cache for private reads.

  • Ignores stampede

    Why it happens. Hot keys turn expiry into a self-DoS on the origin.

    How it fails. Celebrity product TTL hits zero; 5k parallel misses stampede Postgres.

    Fix. Coalesce misses, refresh early, or soft-TTL one filler; mention it whenever you describe hot keys.

4Practise

Concept check

A short quiz on this lesson before open questions. Optional, and it does not gate practice.

Open questions

Ready to practise?

Write an answer, get AI coaching, then revise. You are not building a real system.

Practise with an open question

All systems lessons