Design Twitter for Millions of Users
A microblogging platform with tweet ingestion, hybrid fan-out timelines, typeahead search, and real-time notifications.
Design WhatsApp / Facebook Messenger
A real-time messaging platform supporting 1:1 and group chat, guaranteed delivery to offline recipients via push notifications, end-to-end encryption, and a separate media pipeline for photos, videos, and voice notes.
//The Contract
Requirements
Functionalwhat the system must do
- ✓Users can send and receive 1:1 and group text messages, with delivery to an online recipient typically completing in well under a second
- ✓Messages are never lost when the recipient is offline — they queue until reconnect and additionally trigger a push notification
- ✓Users can send photos, videos, and voice notes; media bytes are uploaded and downloaded independently of the messaging path
- ✓Users see accurate online/last-seen presence for their contacts, subject to privacy settings
- ✓All message content is end-to-end encrypted, so the service operator cannot read message bodies even if its databases are compromised
Non-functionalhow well it must do it
- ≫High availability — sending a message should almost never fail outright, even if delivery to the recipient is momentarily delayed
- ≫At-least-once delivery — a message acknowledged to the sender must never silently disappear, even across server crashes
- ≫Massive scale — billions of daily active users and on the order of 100 billion messages sent per day, the overwhelming majority tiny text payloads
- ≫Low latency — sub-second delivery for the online-to-online path; presence updates should propagate within a few seconds
//Back of the Envelope
Scale Estimates
Rough, order-of-magnitude numbers. The point is to justify the architecture, not to be exact.
Daily active users
2B
collectively sending on the order of 100B messages per day
Messages / second (avg)
~1.2M
spikes several times higher around regional evenings and holidays
Read : write ratio
~1 : 1
unlike a social feed, every message sent is also received — no fan-out read multiplier
Avg text message size
~100 B
tiny compared to media — this is what makes the WebSocket path so cheap at this volume
Online delivery target
< 250 ms
end-to-end target for the WebSocket path when both users are connected
Messages with media
~10%
small share, but large bytes — justifies keeping media off the messaging path entirely
//Follow the Data
Request Flows
A diagram shows what exists; a trace shows what happens. Press play on a flow, or click any step, and the diagram above lights up the exact path that request takes.
Flow 01
Send a 1:1 message (recipient online)
The core real-time path: encrypt, send over a persistent socket, persist, forward.
Flow 02
Deliver to an offline recipient
How the same send path degrades gracefully when the recipient's app isn't running.
Flow 03
Send a photo to a group
Combines the media pipeline with group fan-out, then rejoins the ordinary delivery path.
//Component Breakdown
Key Components
Every box on the canvas, and the job it does. Click a card to locate it in the diagram; click the node itself for the full deep dive.
client×2
network×4
compute×6
database×2
cache×1
queue×1
storage×1
external×1
//Deep Dive
Architecture Breakdown
Architecture Overview
This design models a real-time messaging platform (WhatsApp / Facebook Messenger) built around a persistent-connection core, a durable delivery guarantee, and end-to-end encryption — with media handled as a separate, storage-heavy side path rather than flowing through the messaging tier.
Key Design Principles
Persistent connections, not polling. Every online device holds a long-lived WebSocket to a Chat Server. This is what makes sub-second delivery possible at 1.2M+ messages/second — polling at that scale would multiply request volume by orders of magnitude for mostly-empty responses.
Persist before you forward. A message is written to the Message Store before the Chat Server attempts delivery. This ordering is what turns 'at-least-once' from a slogan into a guarantee: even if the recipient's push fails or the Chat Server crashes mid-delivery, the message already exists durably and can still reach them.
Two delivery paths, one durability mechanism. An online recipient gets the message pushed directly over their existing socket. An offline recipient's copy waits in a per-user partition of the Message Queue until they either reconnect or a Push Worker wakes their device via APNs/FCM — the two paths differ only in the last hop, not in whether the message is safe.
Groups fan out through a service, not a socket loop. The Chat Server never iterates a group's member list itself. It delegates to the Group Service, which resolves membership and publishes one event per member onto the queue — keeping the group send path identical to the 1:1 path from the queue onward, online or offline.
Media never touches the messaging tier. Photos, videos, and voice notes are uploaded and downloaded directly against blob storage via presigned URLs. The message itself only ever carries a storage key — a few bytes — keeping the high-volume messaging path free of multi-megabyte payloads.
End-to-end encryption is a client-side property. The server hosts prekey bundles so clients can perform X3DH key agreement, but it only ever stores and routes ciphertext. Compromising the Message Store does not expose message content.
Fault Tolerance
- Chat Server crash: the client's WebSocket drops, it reconnects with backoff through the Load Balancer to a healthy instance and re-registers in the Session Store; any message already persisted before the crash is not lost.
- Session Store entry expiry: treated as 'offline' by default — worst case is one extra push-notification round trip, not a lost message.
- Push Gateway throttling: the Push Worker retries with backoff and coalesces multiple pending messages into a single notification rather than one push per message.
- Message queue backlog growth: consumers autoscale on lag; a client that falls far enough behind falls back to a full conversation resync instead of replaying an enormous backlog.
//Trade-offs
Design Decisions
An architecture is a record of trade-offs. For every major choice here: what won, what lost, and why the constraints made it so.
01
Real-time transport for message delivery
ChosePersistent WebSocket connection per device
OverClient polling every few secondsLong-polling / Comet
At ~1.2M messages/second with a sub-250ms delivery target, polling would multiply request volume by orders of magnitude for mostly-empty responses. A persistent WebSocket lets the Chat Server push the moment a message arrives, and lets the connection double as the presence heartbeat — one mechanism serves both needs.
02
Tracking which server holds a user's connection
ChoseRedis-backed session/routing table (userId → chat server ID)
OverConsistent-hash routing with no lookup tableSticky sessions enforced only at the load balancer
With billions of devices spread across a large Chat Server fleet, some component must know which instance holds a given socket before a message can be forwarded. A TTL'd Redis lookup is cheap at this read volume and self-heals: if an entry expires because a server died without deregistering, the user is simply — and safely — treated as offline until they reconnect.
03
Guaranteeing delivery to offline recipients
ChoseDurable per-user queue plus a separate push-notification hop
OverDrop the message and rely on the sender retryingKeep retrying the dead WebSocket server-side
Mobile OSes kill background network sockets to save battery, so retrying a WebSocket is pointless once the app is backgrounded. Queueing the message durably (already persisted in the Message Store) and using the platform's own push channel (APNs/FCM) is the only path that reliably wakes the device — and it degrades gracefully since the queue, not the push, is what actually guarantees no message is lost.
04
Message storage engine
ChoseWide-column store partitioned by conversation, clustered by timestamp
OverRelational database (e.g. Postgres) with a single messages tableNo long-term store — keep messages only in the queue
Message history is an append-only, write-heavy, time-ordered workload at a scale (100B+ messages/day) where a single relational table's indexes and row-locking become the bottleneck. Partitioning by conversation and clustering by time gives O(1) access to 'most recent N messages' — exactly the read pattern chat history needs — without cross-partition coordination on writes.
05
End-to-end encryption key exchange
ChoseServer-hosted one-time prekey bundles (Signal Protocol X3DH); server only ever sees ciphertext
OverServer-side encryption at rest only (server can decrypt)Pre-shared keys exchanged out of band
Storing message content unencrypted anywhere in the pipeline means a single database compromise exposes every conversation. Hosting only prekey bundles (public, one-time-use) lets clients bootstrap encrypted sessions without the server ever participating in the actual key material exchange — the Message Store and Message Queue only ever carry bytes the server itself cannot read.
06
Media delivery path
ChosePresigned direct-to-blob-storage upload and CDN-backed download
OverProxy media bytes through the Chat ServerEmbed small media as base64 inside the message payload
Only ~10% of messages carry media, but those bytes are thousands of times larger than a text message. Routing them through the Chat Server would force that tier to scale with media volume instead of message count — the wrong axis given the 1.2M msgs/sec target. Presigned URLs keep the messaging path's payloads at ~100 bytes regardless of how much media the platform moves.
//What Breaks First
Bottlenecks & Failure Modes
Reading a system means sensing where it cracks under 10× load. These are the pressure points of this design, and how it holds.
Risk 01
Reconnect storm after an outage
Failure mode
When a Chat Server fleet redeploys or a regional network blip drops connections, millions of devices attempt to reconnect within the same few seconds, hammering the Load Balancer and re-registering in the Session Store all at once.
Mitigation
Clients reconnect with exponential backoff and jitter rather than immediately. The Session Store is sharded and scaled independently of the Chat Server fleet, and the Load Balancer sheds or queues connection bursts rather than passing them straight through.
Risk 02
Hot conversation partitions
Failure mode
A very large group (hundreds of members) or an unusually chatty 1:1 conversation concentrates writes onto a single Message Store partition and creates a fan-out burst in the Message Queue when the Group Service resolves that group's membership.
Mitigation
Message Store partitions are time-bucketed within a conversation so no single partition grows unbounded. The Group Service caps fan-out concurrency and batches per-member queue writes instead of publishing all N events synchronously.
Risk 03
Push gateway rate limiting
Failure mode
APNs and FCM enforce per-sender rate limits. A burst of offline deliveries — for example right after a regional outage resolves — can get individual push notifications throttled or dropped by the external gateway.
Mitigation
The Push Worker backs off and retries on throttling, and coalesces multiple pending messages for the same user into a single 'you have new messages' notification instead of one push per message.
Risk 04
Message queue backlog growth
Failure mode
If Push Workers or Chat Servers fall behind during a deploy, GC pause, or traffic spike, per-user queue backlogs grow unbounded, delaying delivery for every offline user behind the lag.
Mitigation
Queue consumers autoscale on measured lag. Per-user retention is capped, and a client that reconnects after exceeding it falls back to a full conversation resync from the Message Store instead of replaying an enormous backlog.
//Active Recall
Test Yourself
If you can answer these without scrolling back up, the architecture is yours. Try each one out loud before revealing.
//Go Deeper
Further Reading
- The X3DH Key Agreement Protocolsignal.org
- The Double Ratchet Algorithmsignal.org
- Apache Cassandra Documentationcassandra.apache.org
- Apache Kafka Documentationkafka.apache.org
- Firebase Cloud Messaging DocumentationFirebase Docs
Patterns used in this design
//A Word
From the Creator
Every great system starts as a sketch on a whiteboard. The ability to zoom out and see the whole picture (how services connect, where data flows, what breaks and why) is what separates engineers who build features from engineers who build systems. This diagram is more than boxes and arrows. It's a map of decisions, trade-offs, and intentional design.
Studying architectures isn't just about passing interviews. It's about training your intuition. The more systems you take apart, the better you get at sensing where a monolith will crack, where a queue belongs, or when a cache is hiding a deeper problem. You start seeing patterns instead of chaos.
So keep reading, keep tracing those edges, keep asking "why this way and not that way." The engineers who truly understand large systems are the ones who never stop being curious about how things fit together. That curiosity is the only ingredient that really matters.
Atharva Arbat@arbat_atharva
//Explore