1Understand
The idea
Realtime delivery is how browsers and apps receive server-pushed updates without hammering HTTP on a tight interval. Interviews are not “use WebSockets because trendy”. Pick WebSocket, SSE, or long polling from product needs (bidirectional vs one-way, firewall/proxy behavior), then size connection fan-out separately from REST QPS.
Open sockets change capacity math: millions of idle connections consume file descriptors, memory, and LB state. Sticky routing or a connection gateway owns who holds which socket; heartbeats, reconnect, and replay after blips are part of the design, not polish.
Separate message throughput (events/s through backplane) from connection count (concurrent sockets), both appear in chat and live-score designs.
Intuition
Push vs poll. Polling GET /updates every second multiplies QPS by connected users and wastes bandwidth when nothing changed. Push (server tells client when data arrives) fits live chat, collaborative edits, sports scores, and presence.
WebSocket: full-duplex framed channel over one upgraded HTTP connection. Great for chat, typing indicators, game state. Requires connection-aware infrastructure (sticky sessions or dedicated gateway tier).
SSE (Server-Sent Events): server→client one-way over HTTP; auto-reconnect in browsers; simpler through proxies than raw WebSockets. Fits live feeds, progress bars, “order status changed.”
Long polling: client holds request until event or timeout; fallback when WebSockets blocked. Higher overhead than SSE; still better than 1s blind poll.
Connection ownership. HTTP requests are stateless; sockets are stateful. LB must route reconnects to a gateway that knows channels, or use a pub/sub backplane (Redis, Kafka) so any app node can publish to sockets on any host.
Capacity. A modest API might do 5k RPS; 500k concurrent sockets is a different fleet (RAM, FD limits, kernel tuning). Size connections per host, not just CPU for JSON.
Liveness. Proxies kill idle TCP; ping/pong heartbeats every 30-60s. Reconnect + resume with last_event_id or snapshot+diff
Authorization on subscribe: joining room:{id} must verify membership server-side, never trust client topic names alone.
Broadcast cost: mega-room may use fan-out tree or edge relay instead of naive O(N) loop per message on one host.after disconnect.
What you will be able to do
- Choose WebSocket vs SSE vs long poll for the product
- Design connection routing and reconnect
- Estimate connection count separately from HTTP QPS
When this comes up
- WebSocket
- SSE
- Long polling
- Presence
- Connection limits
2See it
Architecture sketch
WebSocket gateway
Long-lived connections terminate at a gateway; events fan in from the app/queue.
Walkthrough
Push an event
Gateway owns sockets
1 / 4
Clients open long-lived sockets to a gateway tier (sticky by connection).
Read the walkthrough as text
Case, team chat with presence.
1M DAU; ~200k concurrent sockets at peak; messages must appear in <500ms for room members; typing indicators optional.
- Protocol: WebSocket, bidirectional send/receive; SSE insufficient for client-originated messages.
- Connect: client
wss://chat.example.com→ L7 LB with stickiness (or dedicated connection gateway pool) → gateway accepts socket, auth JWT on handshake. - Subscribe: join room channels
room:{id}in gateway memory; store mapping user→socket ids. - Send message: client frame → gateway validates membership → write message to SQL (source of truth) → publish
MessageCreatedto Redis pub/sub channelroom:{id}→ all gateway nodes subscribed deliver to local sockets in that room. - Presence: heartbeat; mark offline after missed pongs; broadcast presence change to room (debounced).
- Reconnect: client exponential backoff; on connect pass
since=messageId; server sends missed messages from DB or short replay buffer. - Scale math: say 50k connections/host × 4 gateway hosts = 200k; monitor FDs and heap; do not size from REST QPS alone.
SSE variant: if product is dashboard “job progress” one-way only → SSE from API tier, no socket cluster complexity.
- Backpressure: slow consumer → drop/coalesce typing events, not unbounded gateway buffer.
- Deploy: drain gateways, stop new connections, wait heartbeat timeout, roll.
Listen for: connection math, backplane, reconnect cursor, not 500ms blind poll.
Napkin math
conn = WebSocket (or SSE) to gateway
1M users × 1 sock ≈ 1M concurrent conns
gateway shards by userId / roomId
events via pub/sub, not sticky app memory
heartbeat every 30s, drop dead sockets
Terminate sockets at the edge. Fan-out through a bus.
3Answer it
How to answer in an interview
When live updates, chat, or “millions watching” appears:
- Delivery need, one-way vs bidirectional; fallback constraints.
- Protocol choice, WebSocket vs SSE vs long poll with reason.
- Connection routing, sticky LB, gateway tier, backplane pub/sub.
- Fan-out model, per-user channel vs room broadcast; who authorizes join.
- Reconnect / replay, last-seen cursor, snapshot.
- Capacity, concurrent connections, heartbeats, per-host limits.
Reject tight HTTP poll unless prompt forces legacy-only clients.
Say: “WebSocket for duplex chat.” · “SSE for one-way feeds.” · “Sticky gateway + Redis pub/sub for multi-node broadcast.”
Answer checklist
Tradeoffs & numbers
- WebSocket: lowest latency duplex; hardest ops (stickiness, gateway fleet).
- SSE: simpler, HTTP-friendly; one-way only.
- Long poll: compatibility; inefficient at huge scale.
- Backplane: needed for multi-node broadcast; Redis single-channel hot spots at mega rooms.
- Stateful tier: harder deploy than stateless APIs, mention draining on rollout.
- Mobile background: push notifications complement sockets when OS
- When NOT WebSocket: rare updates, mobile background, FCM/APNs + occasional fetch.
- Long poll at scale: connection count similar to sockets, still need gateway sizing.
- Single giant room: consider sharded fan-out or read-only CDN for scoreboard static ticks.suspends app.
Kinds of questions (4)
Same pattern, different interview surfaces. Recognise which variant you are in before you answer.
WebSocket chat / presence
Example. Slack-style rooms with typing indicators and online status. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.
How. Duplex WebSocket; room channels; pub/sub backplane across gateway nodes; heartbeat presence. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.
SSE for one-way updates
Example. Server pushes export job progress to the browser. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.
How. SSE HTTP stream; client EventSource; reconnect with Last-Event-ID; no client send on same channel. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.
Long polling fallback
Example. Corporate proxy blocks WebSockets; chat still needs near-realtime. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.
How. Hold GET until event or timeout; client immediately long-polls again; higher overhead than SSE. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.
Connection fan-out at scale
Example. 500k concurrent viewers on a live scoreboard. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.
How. Size gateway fleet by connections/host; Redis/Kafka fan-out; not REST QPS math alone. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.
Common mistakes
Failure modes the AI coach watches for when you practise open questions.
Push vs hammer poll
Avoid. Client polls every 100ms. You invent a DDoS against yourself.
Prefer. WebSocket/SSE/long-poll via a gateway; events from pub/sub.
Weak sketch
Bad
Stronger sketch
Good
Poll everything
Why it happens. Aggressive polling multiplies HTTP QPS by connected users and still lags behind true push.
How it fails. Every client polls `GET /messages?since=` every 500ms for a live chat.
Fix. WebSocket or SSE (or long poll fallback); size push channel and reconnect story.
Ignores connection count
Why it happens. Socket fleets hit file descriptor and memory limits before REST CPU does.
How it fails. Capacity plan only shows 2k API QPS; silent on 300k open WebSockets.
Fix. Estimate concurrent connections, gateways per host, heartbeats; separate from HTTP QPS.
No reconnect story
Why it happens. Networks drop; clients without resume miss messages and show stale UI.
How it fails. Socket closes on tunnel blip; user must refresh to see new chat.
Fix. Backoff reconnect, last-event-id / snapshot+diff, optional offline queue on client.
4Practise
Concept check
A short quiz on this lesson before open questions. Optional, and it does not gate practice.
Open questions
- about 9 min
WebSocket chat design
Design realtime delivery for team chat: clients send messages, receive others’ messages in under 500ms, and show typing indicators. REST QPS is modest but 150k users are concurrently connected at peak. Why WebSocket vs polling, how do connections route through the load balancer, and how do multiple API hosts broadcast to a room?
- about 7 min
SSE progress stream
A web dashboard exports a large report: the server should push progress percentage and final download URL to the browser. The client does not need to send frames on the same channel. Compare WebSocket vs SSE for this case and describe reconnect behavior.
- about 7 min
Long polling fallback
Some enterprise users sit behind a proxy that blocks WebSockets. You still need near-realtime inbox updates within a few seconds. Explain long polling as a fallback: request shape, timeouts, and tradeoffs vs SSE.
- about 8 min
Connection scale math
A live scoreboard expects 400k concurrent viewers receiving score updates every few seconds. Average REST API traffic is only 3k QPS. What capacity dimensions matter for the realtime tier, and how would you fan out an update to all subscribed sockets?
Ready to practise?
Write an answer, get AI coaching, then revise. You are not building a real system.