Network & Geo

Realtime & Connections

Push updates to clients with WebSockets, SSE, or long polling, and size connection fan-out.

Intermediate, about 65 minutes to read, 4 open questions to practise.

1Understand

The idea

Realtime delivery is how browsers and apps receive server-pushed updates without hammering HTTP on a tight interval. Interviews are not “use WebSockets because trendy”. Pick WebSocket, SSE, or long polling from product needs (bidirectional vs one-way, firewall/proxy behavior), then size connection fan-out separately from REST QPS.

Open sockets change capacity math: millions of idle connections consume file descriptors, memory, and LB state. Sticky routing or a connection gateway owns who holds which socket; heartbeats, reconnect, and replay after blips are part of the design, not polish.

Separate message throughput (events/s through backplane) from connection count (concurrent sockets), both appear in chat and live-score designs.

Intuition

Push vs poll. Polling GET /updates every second multiplies QPS by connected users and wastes bandwidth when nothing changed. Push (server tells client when data arrives) fits live chat, collaborative edits, sports scores, and presence.

WebSocket: full-duplex framed channel over one upgraded HTTP connection. Great for chat, typing indicators, game state. Requires connection-aware infrastructure (sticky sessions or dedicated gateway tier).

SSE (Server-Sent Events): server→client one-way over HTTP; auto-reconnect in browsers; simpler through proxies than raw WebSockets. Fits live feeds, progress bars, “order status changed.”

Long polling: client holds request until event or timeout; fallback when WebSockets blocked. Higher overhead than SSE; still better than 1s blind poll.

Connection ownership. HTTP requests are stateless; sockets are stateful. LB must route reconnects to a gateway that knows channels, or use a pub/sub backplane (Redis, Kafka) so any app node can publish to sockets on any host.

Capacity. A modest API might do 5k RPS; 500k concurrent sockets is a different fleet (RAM, FD limits, kernel tuning). Size connections per host, not just CPU for JSON.

Liveness. Proxies kill idle TCP; ping/pong heartbeats every 30-60s. Reconnect + resume with last_event_id or snapshot+diff

Authorization on subscribe: joining room:{id} must verify membership server-side, never trust client topic names alone.

Broadcast cost: mega-room may use fan-out tree or edge relay instead of naive O(N) loop per message on one host.after disconnect.

What you will be able to do

  • Choose WebSocket vs SSE vs long poll for the product
  • Design connection routing and reconnect
  • Estimate connection count separately from HTTP QPS

When this comes up

  • WebSocket
  • SSE
  • Long polling
  • Presence
  • Connection limits

2See it

Architecture sketch

WebSocket gateway

Long-lived connections terminate at a gateway; events fan in from the app/queue.

100%
ClientGatewayServiceQueue

Walkthrough

Push an event

Gateway owns sockets

1 / 4

100%

Clients open long-lived sockets to a gateway tier (sticky by connection).

Read the walkthrough as text

Case, team chat with presence.

1M DAU; ~200k concurrent sockets at peak; messages must appear in <500ms for room members; typing indicators optional.

  1. Protocol: WebSocket, bidirectional send/receive; SSE insufficient for client-originated messages.
  2. Connect: client wss://chat.example.com → L7 LB with stickiness (or dedicated connection gateway pool) → gateway accepts socket, auth JWT on handshake.
  3. Subscribe: join room channels room:{id} in gateway memory; store mapping user→socket ids.
  4. Send message: client frame → gateway validates membership → write message to SQL (source of truth) → publish MessageCreated to Redis pub/sub channel room:{id} → all gateway nodes subscribed deliver to local sockets in that room.
  5. Presence: heartbeat; mark offline after missed pongs; broadcast presence change to room (debounced).
  6. Reconnect: client exponential backoff; on connect pass since=messageId; server sends missed messages from DB or short replay buffer.
  7. Scale math: say 50k connections/host × 4 gateway hosts = 200k; monitor FDs and heap; do not size from REST QPS alone.

SSE variant: if product is dashboard “job progress” one-way only → SSE from API tier, no socket cluster complexity.

  1. Backpressure: slow consumer → drop/coalesce typing events, not unbounded gateway buffer.
  2. Deploy: drain gateways, stop new connections, wait heartbeat timeout, roll.

Listen for: connection math, backplane, reconnect cursor, not 500ms blind poll.

Napkin math

Realtime napkin

conn = WebSocket (or SSE) to gateway

1M users × 1 sock ≈ 1M concurrent conns

gateway shards by userId / roomId

events via pub/sub, not sticky app memory

heartbeat every 30s, drop dead sockets

Terminate sockets at the edge. Fan-out through a bus.

3Answer it

How to answer in an interview

When live updates, chat, or “millions watching” appears:

  1. Delivery need, one-way vs bidirectional; fallback constraints.
  2. Protocol choice, WebSocket vs SSE vs long poll with reason.
  3. Connection routing, sticky LB, gateway tier, backplane pub/sub.
  4. Fan-out model, per-user channel vs room broadcast; who authorizes join.
  5. Reconnect / replay, last-seen cursor, snapshot.
  6. Capacity, concurrent connections, heartbeats, per-host limits.

Reject tight HTTP poll unless prompt forces legacy-only clients.

Say: “WebSocket for duplex chat.” · “SSE for one-way feeds.” · “Sticky gateway + Redis pub/sub for multi-node broadcast.”

Answer checklist

1. Delivery need: push vs client poll. 2. Protocol (WebSocket, SSE, long poll) and why. 3. Connection ownership / sticky LB / gateway. 4. Fan-out: broadcast vs per-user channel. 5. Scale: connections per host, heartbeats, reconnect.

Tradeoffs & numbers

  • WebSocket: lowest latency duplex; hardest ops (stickiness, gateway fleet).
  • SSE: simpler, HTTP-friendly; one-way only.
  • Long poll: compatibility; inefficient at huge scale.
  • Backplane: needed for multi-node broadcast; Redis single-channel hot spots at mega rooms.
  • Stateful tier: harder deploy than stateless APIs, mention draining on rollout.
  • Mobile background: push notifications complement sockets when OS
  • When NOT WebSocket: rare updates, mobile background, FCM/APNs + occasional fetch.
  • Long poll at scale: connection count similar to sockets, still need gateway sizing.
  • Single giant room: consider sharded fan-out or read-only CDN for scoreboard static ticks.suspends app.

Kinds of questions (4)

Same pattern, different interview surfaces. Recognise which variant you are in before you answer.

  • WebSocket chat / presence

    Example. Slack-style rooms with typing indicators and online status. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.

    How. Duplex WebSocket; room channels; pub/sub backplane across gateway nodes; heartbeat presence. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.

  • SSE for one-way updates

    Example. Server pushes export job progress to the browser. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.

    How. SSE HTTP stream; client EventSource; reconnect with Last-Event-ID; no client send on same channel. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.

  • Long polling fallback

    Example. Corporate proxy blocks WebSockets; chat still needs near-realtime. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.

    How. Hold GET until event or timeout; client immediately long-polls again; higher overhead than SSE. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.

  • Connection fan-out at scale

    Example. 500k concurrent viewers on a live scoreboard. Size concurrent sockets separately from REST QPS; heartbeats, reconnect, and sticky routing or connection gateway.

    How. Size gateway fleet by connections/host; Redis/Kafka fan-out; not REST QPS math alone. Gateway holds sockets; backplane for fan-out; replay missed events after reconnect with cursor.

Common mistakes

Failure modes the AI coach watches for when you practise open questions.

Push vs hammer poll

Avoid. Client polls every 100ms. You invent a DDoS against yourself.

Prefer. WebSocket/SSE/long-poll via a gateway; events from pub/sub.

Weak sketch

Bad

100%

Stronger sketch

Good

100%
  • Poll everything

    Why it happens. Aggressive polling multiplies HTTP QPS by connected users and still lags behind true push.

    How it fails. Every client polls `GET /messages?since=` every 500ms for a live chat.

    Fix. WebSocket or SSE (or long poll fallback); size push channel and reconnect story.

  • Ignores connection count

    Why it happens. Socket fleets hit file descriptor and memory limits before REST CPU does.

    How it fails. Capacity plan only shows 2k API QPS; silent on 300k open WebSockets.

    Fix. Estimate concurrent connections, gateways per host, heartbeats; separate from HTTP QPS.

  • No reconnect story

    Why it happens. Networks drop; clients without resume miss messages and show stale UI.

    How it fails. Socket closes on tunnel blip; user must refresh to see new chat.

    Fix. Backoff reconnect, last-event-id / snapshot+diff, optional offline queue on client.

4Practise

Concept check

A short quiz on this lesson before open questions. Optional, and it does not gate practice.

Open questions

Ready to practise?

Write an answer, get AI coaching, then revise. You are not building a real system.

Practise with an open question

All systems lessons