1Understand
The idea
Rate limiting is a fuse in front of expensive or scarce work: APIs, login endpoints, downstream partners, or your own database. It answers *who* is limited (user, IP, tenant, API key), *how fast* they may go, and *what happens* when they exceed the limit; usually 429 with Retry-After.
The limiter must sit before the cost you are protecting, and in a fleet it must share state or you accidentally multiply quotas by instance count. Algorithm choice (fixed window, sliding window, token bucket) is about burst vs steady rate and fairness, not magic.
Say 429 is a contract: clients must backoff; partners integrate Retry-After. Limiters also protect your fleet from retry storms when downstream is slow.
Intuition
Why limit. Abuse (credential stuffing, scraping), fairness (free tier vs paid), and backpressure so one tenant cannot exhaust DB connections or partner API quotas.
Limit key must match identity you actually have: authenticated user id or API key; anonymous traffic → IP (know NAT/shared-IP false positives). Multi-tenant B2B → tenant id, not client IP behind corporate NAT.
Algorithms (interview level):
- Fixed window, count requests per calendar minute; simple but allows 2× burst at window edges.
- Sliding window, smoother; more state (logs or approximate counters).
- Token bucket, refill tokens at steady rate; allows controlled bursts up to bucket size (good for “100/sec steady, burst 20”).
- Leaky bucket, smooths output rate; shapes traffic to downstream.
Placement. Edge API gateway or first middleware before auth-heavy DB work. Rate limit after you have already run the expensive query is “too late.”
Distributed fleets. Counters in Redis (or similar) with atomic increment; local in-memory counters are only OK if the limit is per-instance *by design*. Health checks and internal traffic often bypass limits.
Client contract. 429 Too Many Requests, Retry-After header, optional body with
Global vs local budget: “1000 req/min per API key” is product intent; per-pod counters violate that unless you multiply quota by replicas accidentally.
Adaptive limits: shed load when DB pool > 80%, combine with queue depth, not only request count.quota reset time.
What you will be able to do
- Choose a limit key and window that match the threat
- Pick an algorithm and explain burst vs steady rate
- Place the limiter so it actually protects the expensive resource
When this comes up
- Abuse
- Fairness
- 429 Too Many Requests
- Per-user or per-IP quotas
2See it
Architecture sketch
Limiter in front of expensive work
Gateway checks a shared counter first. Rejected calls never hit the app or DB.
Walkthrough
Allow vs 429
Token bucket on the gateway
1 / 4
Request hits the gateway. Limit key is userId over a 60s window.
Read the walkthrough as text
Case, public REST API with free and paid tiers.
Threat: one API key scrapes the catalog; another shared office IP hits login with many accounts.
- Authenticated catalog reads, limit key =
api_key:{id}; token bucket 100 req/s steady, burst 300; enforced at API gateway after API key validation (cheap) and before DB. - Login, limit key =
ip:{clientIp}+ separateaccount:{username}lockout; low fixed window (e.g. 10/min/IP) because unauthenticated. - Implementation, Redis
INCRwith TTL for sliding-ish windows, or dedicated limiter library; all app instances share Redis so 10 pods do not each multiply quota. - Response, 429 with Retry-After; paid tier higher limits stored in key metadata.
- Bypass, synthetic monitoring uses separate keys with high ceiling.
What a strong answer names. Keys, algorithm (token bucket for burst), gateway placement, shared store, 429 behavior, and why IP limit on login is wrong for catalog reads
- Observability: metric
rate_limit_exceeded_total{route,key_type}, distinguish abuse from misconfigured partner.
- Grace: soft limit (delay) vs hard 429, interviews usually want hard reject before expensive work.behind one NAT.
Napkin math
key = userId (or apiKey)
limit = 100 req / 60s
algo = token bucket (burst 20)
store = Redis INCR + EXPIRE / Lua
deny → 429 + Retry-After
Limit before the expensive path. Shared store so N replicas do not multiply quota.
3Answer it
How to answer in an interview
When the prompt mentions abuse, quotas, or protecting downstreams:
- Resource protected, DB, partner API, email send, expensive ML call.
- Identity / key, user, API key, IP, tenant; justify for auth model.
- Limit shape, requests per second/minute/day; burst allowance.
- Algorithm, tie to burst vs steady (token bucket is the usual hero).
- Where it runs, gateway vs service middleware; before expensive work.
- Distributed, shared Redis vs per-node mistake.
- Client UX, 429, Retry-After, differentiation of tiers.
One sentence on what you would *not* limit (health, static assets)
Say: “Enforce at gateway after cheap auth, before DB.” · “Token bucket for burst + steady.” · “Shared Redis counters for fleet-wide quota.”shows maturity.
Answer checklist
Tradeoffs & numbers
- Strict global limits: fair but need shared state; Redis becomes a dependency, cache/local fallback risks over-admission.
- Approximate local limits: survive Redis blip but effective quota scales with replica count, state the risk.
- IP limits: cheap for login abuse; unfair behind carrier-grade NAT.
- Sliding vs fixed window: sliding is fairer; fixed is simpler at very high QPS.
- Reject vs queue: rate limits usually reject fast (429); queuing is a different pattern (backpressure queue).
- False negatives: whitelisting partners; cost of bypass holes, monitor anomalously high 429 rates.
- When NOT to rate-limit: internal mesh with mTLS and fixed callers, use quotas at integration boundary instead.
- Fixed window at 100k QPS: edge doubling at minute boundary, prefer sliding or token bucket for public API.
- Cost of Redis limiter: extra RTT per request, acceptable at gateway, not inside inner loop per row.
Kinds of questions (4)
Same pattern, different interview surfaces. Recognise which variant you are in before you answer.
Per-user API quota
Example. Paid API customers get higher limits than free tier. Specify limit key (user, API key, IP, tenant), steady vs burst allowance, HTTP 429 + Retry-After, and where enforcement sits before the DB or partner call.
How. Key = API key or user id; token bucket per tier; 429 + Retry-After; enforce at gateway before DB. Enforce at gateway; return 429 early; distributed store if fleet >1; never rate-limit after the expensive query.
IP-based abuse shield
Example. Unauthenticated login endpoint hit by credential stuffing. Specify limit key (user, API key, IP, tenant), steady vs burst allowance, HTTP 429 + Retry-After, and where enforcement sits before the DB or partner call.
How. Key = client IP (and optionally username); low fixed window; complement with CAPTCHA/lockout, not for NAT-heavy catalog.
Token bucket bursts
Example. Clients poll status every second but may burst 50 at job start. Specify limit key (user, API key, IP, tenant), steady vs burst allowance, HTTP 429 + Retry-After, and where enforcement sits before the DB or partner call.
How. Steady refill rate + bucket size; explain burst allowance vs fixed window edge doubling. Enforce at gateway; return 429 early; distributed store if fleet >1; never rate-limit after the expensive query.
Distributed limiter with Redis
Example. Fleet of 20 API servers must share one global quota. Specify limit key (user, API key, IP, tenant), steady vs burst allowance, HTTP 429 + Retry-After, and where enforcement sits before the DB or partner call.
How. Central counter in Redis with atomic ops; never per-instance memory-only for global limits. Enforce at gateway; return 429 early; distributed store if fleet >1; never rate-limit after the expensive query.
Common mistakes
Failure modes the AI coach watches for when you practise open questions.
Where the counter lives
Avoid. Per-instance memory counter. 10 replicas ≈ 10× the quota.
Prefer. Shared Redis (or edge) limiter. Fleet size does not multiply allowance.
Weak sketch
Bad
Stronger sketch
Good
Wrong limit key
Why it happens. The wrong identity dimension blocks innocents or misses abusers.
How it fails. Per-IP limit on a mobile app API where thousands of users share one carrier NAT.
Fix. Match key to auth: API key/user id when authenticated; IP for login abuse; tenant id for B2B.
Too late
Why it happens. Limiting after the expensive query already ran does not protect the database.
How it fails. Rate limit checked inside the handler after a heavy join and external API call.
Fix. Enforce at gateway or first middleware before DB/partner calls; reject with 429 early.
Local-only in a fleet
Why it happens. Per-process counters multiply the effective limit by instance count.
How it fails. 100 req/s in-memory per pod × 20 pods ≈ 2000 req/s effective for one API key.
Fix. Shared store (Redis) or explicit per-instance limits only if product intent is per-node.
4Practise
Concept check
A short quiz on this lesson before open questions. Optional, and it does not gate practice.
Open questions
- about 7 min
Per-user API quota
You ship a public API with free (1k req/day) and paid (100k req/day) tiers. API keys identify customers. Where do you enforce limits, what algorithm/window do you use, and what HTTP response do clients get when over quota?
- about 7 min
Login abuse shield
Unauthenticated POST /login is hammered by credential stuffing. You cannot rely on user id yet. How do you rate limit, what key do you use, and what are the downsides for users behind shared NAT?
- about 7 min
Token bucket bursts
A client may send steady traffic at 50 req/s but occasionally bursts 200 requests in one second when a batch job starts. Compare fixed window vs token bucket for this shape. Which do you pick and why?
- about 8 min
Fleet-wide limiter
Twenty API servers behind a load balancer must enforce 1000 req/s total per API key (not 1000 per server). How do you implement the counter, where does it live, and what breaks if Redis is briefly unavailable?
Ready to practise?
Write an answer, get AI coaching, then revise. You are not building a real system.