Design a Chat Application

Requirements

Functional: one-to-one messaging; message delivery even if the recipient is offline (queued for later); delivery/read receipts; online/offline presence. Non-functional: low latency for online users; must not lose messages if a server restarts mid-conversation; must scale to millions of concurrent connections.

Why HTTP polling doesn't work here

Repeatedly polling "do I have new messages?" every few seconds wastes bandwidth and adds latency proportional to the poll interval. Real-time delivery needs a persistent connection — WebSockets (or long-polling as a fallback) — so the server can push a message the instant it arrives, instead of the client having to ask.

Architecture

Why presence tracking matters architecturally

With many gateway server instances behind a load balancer, User A's message can't just be broadcast to "the WebSocket layer" — the system needs to know the exact server instance holding User B's specific connection to route the message correctly. This is why a separate Presence Service (usually backed by Redis, mapping user_id → gateway_instance_id) is a distinct component, not an afterthought.

Follow-up: how do you handle message ordering and duplicate delivery?

Each message gets a monotonically increasing ID (per-conversation) so clients can detect and discard duplicates or reorder out-of-sequence deliveries — this matters because retries (from a gateway crash mid-delivery, for example) can otherwise cause the same message to be shown twice.