This is my solution to the homework from the news feed session: design a Discord-style messaging system. Unlike a feed, chat must deliver within a few hundred milliseconds over always-open connections, keep order within each channel, and store history forever. Afterwards you’ll be able to present the design from gateway to storage, plus the hard cases like a server with a million members.
1. Clarify the requirements
Key idea: pin the scope: servers (guilds) with many channels, direct messages (DMs), real-time delivery, scrollable history, online status, unread counts. Voice, video and large files are out of scope.
Features:
- Users join servers; servers have many text channels; there are one-to-one DMs and small group DMs.
- Send a message and every online member of the channel sees it in < 500 ms.
- Scroll back through unlimited history, edit and delete messages.
- Presence: online/offline/idle. Typing indicators are a bonus.
- Unread counts per channel, and @mentions trigger a push notification when offline.
Non-functional: delivery latency p99 < 500 ms; correct order within a channel; never lose a message once “sent” is confirmed; highly available; servers up to 1 million members.
2. Estimate
Key idea: the expensive part isn’t message writes (35K/s) but deliveries (each message reaches tens to hundreds of thousands of people) and always-open connections (10M).
Assumptions (state them): 150M MAU, 30M DAU, 10M concurrent connections at peak, 100 messages per DAU per day, 300 bytes per message including metadata, each message reaches 20 online people on average.
- Message writes: 30M × 100 = 3 × 10⁹/day ÷ 10⁵ ≈ 30–35K/s, peak ×3 ≈ 100K/s.
- Deliveries: 35K × 20 ≈ 700K deliveries/s, ~2M/s at peak.
- Storage: 3 × 10⁹ × 300 B ≈ 900 GB/day ≈ 330 TB/year; ×3 replicas ≈ 1 PB/year. Nothing is deleted, so storage must grow by adding nodes online.
- Connections: 10M WebSockets; one gateway node holds ~50K connections (a few tens of KB of memory each) → ~200 nodes. Heartbeats every 40 seconds → 10M / 40 ≈ 250K heartbeats/s.
3. High-level architecture
Key idea: separate the send path (REST, stateless, easy to scale) from the receive path (WebSocket, holding connection state). In between, a message service writes to ScyllaDB and pushes events through a fan-out layer that knows who’s online and where.
flowchart LR
C[Client] -->|POST message| API[API / Message service]
API --> DB[(ScyllaDB: messages)]
API --> K[Kafka: keyed by guild_id]
K --> GS[Guild service: who is online, on which node]
GS --> G1[Gateway node 1]
GS --> G2[Gateway node N]
G1 -->|WebSocket| C
GS -.->|offline + mention| PN[Push notification]
C -->|WebSocket: heartbeat, presence| G1
- Gateway: holds WebSockets, authenticates on connect, receives heartbeats, pushes events to clients. No business logic. Epoll-style I/O so one node can hold tens of thousands of connections (see I/O models and epoll).
- Message service: checks permissions (cached), assigns
message_id, writes to the DB, publishes an event to a Kafka partition keyed byguild_id(keeping order within a guild). - Guild service: sharded by
guild_idwith consistent hashing. Each instance keeps in memory the online sessions of the guilds it owns, and which gateway node each session is on. - Session registry: Redis
session:{user_id} → {gateway_node, session_id}, TTL refreshed by heartbeats.
4. Data model and message order
Key idea: reading history is always “the 50 messages before message X in channel Y”, so partition by channel and time window and cluster by message_id descending. Snowflake gives you order and a cursor for free.
-- ScyllaDB (CQL)
CREATE TABLE messages (
channel_id bigint,
bucket int, -- 10-day window number, derived from the timestamp in message_id
message_id bigint, -- Snowflake
author_id bigint,
content text,
edited_at timestamp,
deleted boolean,
PRIMARY KEY ((channel_id, bucket), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);
CREATE TABLE read_states ( -- what each user has read
user_id bigint, channel_id bigint, last_read_id bigint, mention_count int,
PRIMARY KEY (user_id, channel_id)
);
Guilds, channels, roles and permissions are relational and rarely written: keep them in PostgreSQL and cache them heavily. The member list of a large guild (1M rows) lives in ScyllaDB, partitioned by (guild_id, bucket).
Reading history: SELECT ... WHERE channel_id = ? AND bucket = ? AND message_id < ? LIMIT 50. When the current bucket runs out, step back to the previous one. Quiet channels have many empty buckets: store the channel’s last_message_id to jump straight there.
Order: a Snowflake id is time + machine + sequence, so messages in a channel sort by message_id. Two messages sent in the same millisecond from two machines get an arbitrary but consistent order for everyone, which is enough for chat. Clients always render by message_id, never by arrival order.
5. Sending and delivering a message
Key idea: the client sends a nonce; the server writes to the DB before confirming; fan-out goes to each gateway node that has a recipient; clients dedupe by message_id.
sequenceDiagram
participant A as Client A
participant M as Message service
participant D as ScyllaDB
participant K as Kafka
participant G as Guild service
participant N as Gateway node
participant B as Client B
A->>M: POST /channels/42/messages (nonce)
M->>D: INSERT (QUORUM)
M-->>A: 200 message_id
M->>K: MESSAGE_CREATE
K->>G: event by guild
G->>N: one copy per node with recipients
N->>B: WebSocket dispatch
Details worth saying:
- Idempotent sends: flaky networks make clients resend. The client generates the
nonce, and the server remembers(user_id, nonce) → message_idfor a few minutes; a resend returns the existing message instead of creating a new one (see idempotency). - Write first, publish second: only confirm once the DB has it at
QUORUM. If publishing to Kafka fails after the write, other clients still see the message when they load history; for stronger guarantees, use the outbox pattern. - Permission checks before delivery: the guild service filters out members who can’t see the channel (role-based permissions, cached per guild).
- Offline: offline members who are @mentioned or get a DM go through push notifications (APNs/FCM). Unread counts are computed on app open by comparing
read_stateswith each channel’slast_message_id.
6. A server with a million members
Key idea: never fan out per user over the network. Group by gateway node, deliver only to people who are online and “subscribed” to that channel, and cut presence traffic by only sending what the user can actually see.
- Fan-out per node: 100K online members sit on ~200 gateway nodes. The guild service sends 200 messages, each carrying a list of sessions; each node loops over its own connections. Internal network cost drops ~500×.
- Only active viewers: the client reports which channel is open; typing and presence events go only to people with that channel open. New messages go to everyone who can see the channel so unread markers update, but can be collapsed into a lighter event.
- Lazy presence: a 1M-member server doesn’t broadcast every online/offline change to everyone. It only sends presence for the members currently shown in the client’s (paginated) member list.
- Slowmode and rate limits: cap messages per user per channel so one channel can’t become a storm.
- A hot guild: if one guild service instance can’t keep up with a guild, split that guild into shards by channel.
7. Connections, presence and reconnects
Key idea: the client sends heartbeats at an interval set by the gateway; missing them closes the session and marks the user offline. A reconnect within a few minutes resumes and replays missed events; longer than that, the client reloads from last_read_id.
- Heartbeats roughly every 40 seconds. If the gateway misses them, it treats the connection as lost and changes presence after a grace period, so mobile users on flaky networks don’t flicker online/offline. This is exactly the liveness and timing wheel problem: 10M sessions, one timing wheel per gateway node for its own sessions.
- Resume: every event sent down has a sequence number within the session. The client reconnects with its
session_idand the last number received; the gateway (or a few-minute event buffer in Redis) replays what’s missing. - Can’t resume: the client calls
GET /channels/{id}/messages?after=<last id>for open channels. With Snowflake ids as the cursor, nothing is missed or duplicated. - A gateway node dies: 50K clients lose their connection at once and reconnect to other nodes. Add random jitter to the retry delay so they don’t all hit at once (a thundering herd); see timeouts, retries and circuit breakers.
Supporting pieces: search, edits and deletes, attachments, encryption
- Search: the message service emits events and a consumer indexes them into Elasticsearch in batches (see Elasticsearch). Index per guild so a query only touches that guild’s shard. A few seconds of search lag is fine.
- Edits and deletes: update the row (
edited_at,deleted = true) and publishMESSAGE_UPDATE/MESSAGE_DELETE. Soft delete so clients with history open receive the event; the data is purged later per policy. - Attachments: the client requests a pre-signed URL, uploads straight to object storage, and the message only holds a link. A CDN serves downloads.
- DMs: a DM is a “channel” with two members and reuses the whole path above; there’s no guild, so fan-out looks up the session registry directly.
- End-to-end encryption: if E2EE is required, the server can’t read content, so server-side search and content moderation become impossible. Discord doesn’t use E2EE for text by default; frame it as a product trade-off.
- Moderation: a separate consumer scans content (spam, malicious links), off the send path so it adds no latency.
8. Presenting it in 45 minutes
Key idea: spend the most time on the three things the interviewer wants to hear: the message data model, delivery through the gateway, and huge servers.
| Minutes | Step | Points to make |
|---|---|---|
| 0–5 | Requirements | guilds, channels, DMs, history, presence; voice out of scope |
| 5–10 | Estimates | 35K writes/s, 700K deliveries/s, 10M connections, 1 PB/year |
| 10–17 | API + data model | REST to send + WebSocket to receive, messages table with buckets, Snowflake |
| 17–27 | Send and deliver path | nonce, QUORUM write, Kafka by guild, guild service, gateway |
| 27–37 | Huge servers, presence | fan-out per node, lazy presence, coalescing |
| 37–42 | Failures | gateway dies, resume, backfill with a cursor, thundering herd |
| 42–45 | Recap | three decisions: split send/receive, partition by channel + time, fan-out per node |
Diagramming and presenting tips are in the URL shortener post.
Remember
- Can you explain why the send path (REST) and the receive path (WebSocket gateway) are separate?
- Can you write the
messagestable with a(channel_id, bucket)partition and explain the role of Snowflake ids? - Can you trace a message from the moment it’s sent to the moment it appears on a recipient’s screen?
- Do you know the three techniques for a million-member server: fan-out per node, lazy presence, read coalescing?
- Do you know what the client does after losing its connection for a few seconds versus a few hours?
Interview questions
Mid
Why does chat use WebSockets instead of the client polling the server?
A WebSocket keeps one two-way connection open so the server pushes messages the moment they exist; polling is either late (infrequent) or wasteful (frequent, mostly empty responses).
With 10M people online, polling every 2 seconds is 5M requests/s, mostly useless. Long polling and SSE are fallbacks when WebSockets are blocked.
Design the message table for scrolling back through history.
Partition by (channel_id, bucket) where the bucket is a time window, cluster by message_id descending; the next page is message_id < cursor LIMIT 50.
The bucket keeps partitions from growing forever. message_id is a Snowflake id, so it’s unique and time-sorted.
A flaky network makes the client send a message twice. How do you avoid showing it twice?
The client sends a nonce; the server remembers (user_id, nonce) → message_id for a few minutes and returns the existing message on a repeated nonce.
On the receiving side, clients dedupe by message_id, since delivery is at-least-once.
Senior
A message lands in a channel of a server with 100K people online. How do you deliver it cheaply?
Fan out per gateway node: the guild service knows which node each session is on, sends one copy plus a session list to each node (~200), and each node delivers to its own connections.
Add lazy presence, send typing only to people with the channel open, and coalesce identical history reads in front of the database.
How do you guarantee message order within a channel?
Assign a Snowflake message_id at write time and always render by id, never by arrival order; publish to a Kafka partition keyed by guild to keep delivery order within a guild.
Two messages in the same millisecond get an arbitrary but consistent order for everyone. On a gap, the client backfills with the cursor.
A gateway node holding 50K connections dies. What happens and how do you limit the damage?
50K clients reconnect to other nodes; use backoff with jitter to avoid a stampede, resume with the session id and last event number, and backfill with ?after= if resume fails.
Thanks to the presence grace period, friends don’t see a wave of people going “offline” and back “online”.
Why ScyllaDB/Cassandra for messages, and what do you have to accept?
Because the access pattern is heavy writes and reads by channel and time: LSM writes are cheap, partitioning and replication are built in, and nodes are added online for data growing ~1 PB/year.
You accept: queries only by partition key, no joins, tombstones on delete, bounding partition size yourself with buckets, and choosing a consistency level per query.
Useful phrases for interviews
- “I’d split the send path from the receive path: REST in, WebSocket out.”
- “The expensive part isn’t writes, it’s deliveries and open connections.”
- “Messages are partitioned by channel and a ten-day bucket to keep partitions bounded.”
- “Snowflake ids give us ordering and a pagination cursor for free.”
- “Delivery is at-least-once, so the client dedupes by message id.”
- “We fan out per gateway node, not per user.”
- “Presence is lazy in large guilds: you only get updates for members you can see.”
- “On reconnect, the client resumes the session or backfills with an after-cursor.”
- “Jittered backoff keeps fifty thousand reconnects from arriving at the same instant.”
- “End-to-end encryption would rule out server-side search, so it’s a product decision.”
Notes based on the System Design course (Ronin Engineer), rewritten in my own words.