The news feed is a classic system design question because it forces you to choose between expensive writes and expensive reads. This post walks through a full interview for a Threads-like social network: requirements, arithmetic, the hybrid push/pull model, and details like celebrities, inactive users and like counters. Read the URL shortener post first if you want the full 45-minute presentation framework.
1. Clarify the requirements
Key idea: pin down five features (follow, post, notify, feed, react) and ask the question that drives the design right away: which posts do users care about most.
Features:
- B follows A.
- A publishes a post, text only.
- B is notified when A posts.
- Every user has a feed, newest first (reverse chronological), no ML ranking yet.
- B reacts to posts: like, comment, share; show the reaction counts.
| Question | Answer |
|---|---|
| How many users, how many DAU? | 500M users, 50M DAU |
| Posts per user per day? | 1 post per DAU |
| Average comments per post? | 10 |
| Which posts do users care about most? | posts from the last 7 days and trending posts |
The last one matters most: it lets you cache only 7 days and ignore old data when building a feed.
Non-functional: return the feed under 100 ms; new posts reach followers’ feeds within seconds; scalable; highly available; eventual consistency is fine (B’s feed lagging A’s post by a few seconds is okay).
2. Estimate
Key idea: posting is light (~580/s), feed reads are 10× heavier, and the most expensive thing is fan-out: 50M posts × 200 followers = 10 billion inbox writes a day.
Extra assumptions (say them out loud): each DAU opens the feed 10 times a day, 200 followers on average, 3 KB per post (from the slides, generous for text only), 500 bytes per comment.
- Posts: 5 × 10⁷ / 10⁵ ≈ 500–580/s, peak ×3 ≈ 1.7K/s.
- Comments: 10 × 50M = 500M/day ≈ 5.8K/s.
- Feed reads: 50M × 10 = 5 × 10⁸/day ≈ 5.8K/s, peak ×5 ≈ 30K/s. Each read pulls 20 posts, so ~600K post reads/s at peak, which must come from cache.
- Fan-out if you push everything: 5 × 10⁷ × 200 = 10¹⁰ inbox writes/day ≈ 116K/s. Pushing only to active users (say 20% of followers) brings it to ~23K/s.
- Storage over 5 years: a post with its comments = 3 + 10 × 0.5 = 8 KB; 8 KB × 5 × 10⁷ × 1,825 days ≈ 730 TB (the slides round to 2,000 days and get 800 TB); ×3 replicas ≈ 2.2 PB.
- 7-day post cache: 3 KB × 5 × 10⁷ × 7 ≈ 1 TB, a Redis cluster of a few dozen nodes.
- Feed cache: 200 post ids × 8 bytes ≈ 1.6 KB per user; 50M DAU ≈ 80 GB (Redis structure overhead can multiply that a few times).
3. API and data model
Key idea: a feed is an ordered list of ids per user, paginated with a cursor. Every table is partitioned by exactly what you look it up by: posts by post_id, feeds by user_id, outboxes by author_id.
POST /api/v1/posts { "content": "..." } → 201 { "postId": "..." }
GET /api/v1/feed?cursor=<postId>&limit=20 → 200 { "items": [...], "nextCursor": "..." }
POST /api/v1/users/{id}/follow → 204
POST /api/v1/posts/{id}/reactions { "type": "LIKE" } → 204
GET /api/v1/posts/{id}/comments?cursor=...
The feed uses a cursor (the id of the last post seen), not an offset: new posts inserted at the top shift the offset and cause repeats (see advanced REST API design).
-- ScyllaDB (CQL). post_id is a Snowflake id: grows with time, so it sorts.
CREATE TABLE posts (post_id bigint PRIMARY KEY, author_id bigint, content text,
status text, created_at timestamp, updated_at timestamp);
CREATE TABLE user_posts (author_id bigint, post_id bigint, -- the author's outbox
PRIMARY KEY (author_id, post_id)) WITH CLUSTERING ORDER BY (post_id DESC);
CREATE TABLE feed (user_id bigint, post_id bigint, author_id bigint, -- the reader's inbox
PRIMARY KEY (user_id, post_id)) WITH CLUSTERING ORDER BY (post_id DESC)
AND default_time_to_live = 2592000; -- keep 30 days
CREATE TABLE followers (user_id bigint, follower_id bigint, PRIMARY KEY (user_id, follower_id));
CREATE TABLE following (user_id bigint, followee_id bigint, PRIMARY KEY (user_id, followee_id));
The slides have an order column in the Feed table; using a Snowflake post_id as the clustering key gives you the order for free. Users and subscriptions live in PostgreSQL.
4. Push, pull or hybrid
Key idea: push moves the cost to write time so reads are cheap; pull moves it to read time so writes are cheap. Neither survives alone with both regular users and celebrities, so go hybrid.
Push (fan-out on write). Each user has an outbox (their own posts) and an inbox (posts from people they follow). Posting drops a letter into the outbox, then the system, like a postman, copies the post id into every follower’s inbox. Reading the feed is just reading the inbox.
Pull (fan-out on read). No inbox. When the feed opens, fetch the list of people you follow, read the latest posts from each one’s outbox, merge and sort.
| Push | Pull | |
|---|---|---|
| Reading the feed | very fast, one inbox read | slow: N outbox reads, then merge |
| Publishing | expensive: write to every follower’s inbox | cheap: one write |
| Celebrity with 100M followers | 100M writes for one post, minutes of delay | no problem |
| Inactive users | wasted work and space for people who never read | costs nothing |
| Unfollow | old posts must be cleaned from the inbox | correct by construction |
| Following topics (tech, travel) | fan-out by topic too | easy to add sources |
The slides conclude “pick pull because it scales”, then the final design is hybrid. I’d say it plainly: go hybrid, because pure pull can’t hit 100 ms for someone following 500 people.
flowchart LR
A[Author posts] --> PS[Post service]
PS --> PDB[(posts, user_posts)]
PS --> K[Kafka: post-created]
K --> FB[Feed builder]
FB -->|regular author, active followers| IN[(Inbox: Redis + feed table)]
FB -.->|celebrity| X[No fan-out, only cache recent posts]
R[Reader opens feed] --> FS[Feed service]
FS --> IN
FS -->|pull new posts from followed celebrities| CC[(Recent celebrity posts cache)]
FS --> H[Hydrate: content, author, counters]
The write path: the feed builder
- The post service stores the post, writes
user_posts, and publishes apost-createdevent to Kafka. - The feed builder consumes it and asks the relationship service: is the author regular or a celebrity (say > 100K followers)?
- Regular: page through followers 1,000 at a time, keep only active ones, write the post id into their inboxes (Redis
LPUSH+LTRIMto keep 200 entries, plus thefeedtable). - Celebrity: no fan-out. Just push the post into the “recent celebrity posts” cache.
The read path: merge and hydrate
- Read the precomputed inbox (the 200 newest ids) from Redis.
- Fetch the celebrities you follow (usually few, a few dozen) and read their latest posts from cache.
- Merge the sorted lists (k-way merge) and take the first 20 ids after the cursor.
- Hydrate:
MGETpost content, author info and counters from cache; drop deleted posts and authors you’ve unfollowed.
The 100 ms budget: inbox read ~2 ms, celebrity posts ~5 ms, parallel hydration ~10 ms. Plenty left for network and serialisation.
Write path details: paging followers, idempotency and priority
- Splitting work: a
post-createdevent from an author with 50K followers spawns 50 sub-tasks (1,000 followers each) on afanout-taskstopic. Many workers run in parallel; if one dies, Kafka hands its task to another. - Idempotency: inbox writes are keyed by
(user_id, post_id), so redelivery doesn’t duplicate. With a Redis list, check a side set first, or use a ZSET with score = post_id (ZADD is naturally idempotent). - Priority: fan out to online users first (from presence), then offline but still “active” users.
- Backpressure: when workers fall behind (Kafka lag grows), feeds get slower but nothing is lost; track lag as an SLO, “a post appears in feeds within 5 seconds”.
- Unfollow: don’t clean inboxes right away. During hydration, drop posts from people no longer followed using the cached
followingset. A background job cleans up later. - Deleting a post: flip
status; hydration drops anything notACTIVE. No need to touch 10 million inboxes.
5. Tracking active users
Key idea: fanning out to people who never open the app is waste. Mark users “active” with a heartbeat when they open the app, rate-limit it with Redis, and let a segmentation service flip their status after 7 silent days.
- Heartbeat = opening the app or refreshing the feed. What “active” means is a business decision (logged in within 7 days, for example).
- Too many hits: 50M DAU × dozens of times a day. Gate it with
SET user:active:{id} <ts> NX EX 86400: only the first hit of the day goes further, the rest stop at Redis. - The first hit of the day emits an event to a
user-statustopic; the segmentation service updates the active set. After 7 days without an event the user becomes “inactive” (this is the liveness and timing wheel problem at a daily scale). - A user returning after a long break has an empty inbox. Build the first feed by pulling, write it into the inbox, then they receive pushes as normal.
6. Reactions, counters, notifications and trending
Key idea: a reaction is stored as one row keyed by (post_id, user_id) so it can’t be counted twice, while the displayed count flows through Kafka and is batched before being added to Redis.
- Writing a reaction: table
reactions (post_id, user_id, type): liking twice is still one row. Publish an event to areactionstopic. - The counter service batches events for 1 second or 10 events, then does one
HINCRBY post:counters:{id} likes 37. A viral post with 50K likes/s turns into a few thousand Redis commands. - Periodically copy counters from Redis to the DB. A displayed count that lags by a few seconds or a few units is fine.
- Notifications: the notification service consumes
post-created. For celebrities, send in rate-limited batches, prioritising users who turned on notifications for that author. - Trending: a streaming job scores engagement with time decay, for example
(likes + 2·comments + 3·shares) / (age_hours + 2)^1.5, and keeps the top-k in a ZSET. A rule engine or an ML model tags topics (tech, business, health) for per-topic trending.
7. The overall design
Key idea: each service owns its data and they talk asynchronously through Kafka. The feed builder is the heart of the write path; the feed service is the heart of the read path.
flowchart TB
C[Client] --> GW[API gateway]
GW --> PS[Post service]
GW --> FS[Feed service]
GW --> RS[Reaction service]
GW --> REL[Relationship service]
GW --> LV[Liveness service]
PS --> KP[Kafka: post-created]
KP --> FB[Feed builder]
KP --> NS[Notification service]
FB --> FD[(Feed: Redis + ScyllaDB)]
FS --> FD
RS --> KR[Kafka: reactions]
KR --> CS[Counter service]
CS --> RC[(Redis counters)]
LV --> KU[Kafka: user-status]
KU --> SG[Segmentation service]
FB --> SG
FB --> REL
On where to store follow relationships: the slides use a graph database (Neo4j). For the feed you only need one hop, “who follows A” and “whom does A follow”, so two adjacency tables (followers, following) on ScyllaDB are enough and scale more easily. A graph DB earns its place for multi-hop “friends of friends” recommendations.
Failure scenarios and how the system reacts
| Incident | Impact | Response |
|---|---|---|
| Feed builder falls behind (Kafka lag) | new posts appear late | add workers, prioritise online users; feeds still readable |
| One shard of Redis inboxes is lost | users on that shard see an empty feed | rebuild from the feed table in ScyllaDB, or pull |
| A celebrity posts and millions open it at once | that post’s key gets hot | a few seconds of in-process cache, replicate the hot key |
| Relationship service is slow | feed builder can’t tell regular from celebrity | cache author classification, default to celebrity (no fan-out) when unsure |
| Counter service dies | like counts freeze | events stay in Kafka and are processed on restart; counts are eventually right |
| Duplicate notification | user gets two notifications | idempotency key on (user_id, post_id) with a TTL |
8. Presenting it in 45 minutes
Key idea: 5 minutes of requirements, 5 of estimates, 10 of API and data model, 15 on push/pull/hybrid (the part the interviewer wants to hear), 7 on failures and scaling, 3 to recap.
- 0–5: requirements. Ask early “which posts do users care about” and “is the feed chronological or ranked”.
- 5–10: estimates. State the 116K writes/s fan-out figure: it’s the reason the whole problem exists.
- 10–20: API (cursor), tables
posts,user_posts,feed,followers. - 20–35: draw push, raise the celebrity problem; draw pull, raise latency; settle on hybrid, the write path and read path, active users.
- 35–42: counters, notifications, trending, incidents.
- 42–45: the three main decisions: hybrid fan-out, feeds store ids and hydrate, push only to active users.
Diagramming conventions and presenting tips are in the URL shortener post. This session’s homework (designing a Discord-style chat system) has its own worked solution in designing a chat system.
Remember
- Can you derive feed reads, posts and fan-out per second from DAU, and point out where the slides are wrong?
- Can you compare push and pull on six criteria and explain why you’d go hybrid?
- Can you draw the hybrid write path (feed builder) and read path (merge, hydrate)?
- Do you know how to track active users without overloading the system?
- Do you know how to count likes on a viral post without a hot row or hot key?
Interview questions
Mid
What’s the difference between fan-out on write and fan-out on read?
On write (push): publishing copies the post id into every follower’s inbox, so reading is very fast. On read (pull): opening the feed gathers posts from everyone you follow, so writes are cheap but reads are slow.
Push breaks on celebrities and inactive users; pull breaks on latency when you follow many people. In practice you go hybrid.
Why does the feed store only post ids instead of the whole post?
Because one post sits in thousands of inboxes: storing content multiplies memory by thousands, and every edit or delete would have to update thousands of places.
Store the id (8 bytes) and hydrate content, author and counters from cache with one MGET at read time. Deleted posts are filtered during hydration.
Why paginate the feed with a cursor rather than an offset?
Because new posts keep landing at the top: with an offset, page 2 shifts and users see posts again or miss some.
The cursor is the last post id returned; the next page takes ids smaller than it. With Snowflake post ids, that’s time order.
Senior
A celebrity with 100 million followers posts. What does your system do?
No fan-out. The post only goes into the “recent celebrity posts” cache; when a follower opens their feed, the feed service pulls it and merges it with their inbox.
Notifications go out in rate-limited batches. That post’s key will be very hot, so add an in-process cache and replicate the key.
Design the feed read path to stay under 100 ms.
Read the precomputed inbox from Redis, read new posts from followed celebrities from cache, k-way merge to 20 ids, then hydrate in parallel with MGET.
No database read on the hot path. If the inbox is empty (a returning user), build it once by pulling and store it.
A user unfollows A. What happens to A’s posts in their inbox?
Don’t delete them immediately; during hydration, drop posts from authors no longer in the (cached) following set, and let a background job clean the inbox later.
Synchronous deletes across inboxes are unnecessary writes; filtering on read is cheap because the following set is already cached.
How do you cut fan-out cost for hundreds of millions of inactive users?
Track active users with a heartbeat on app open (gated by SET NX EX once a day), push only to active ones, and build the feed by pulling once for anyone returning after a long break.
Fan-out shrinks by the share of active users; the price is a liveness service, a segmentation service and a cold-start path.
Like counters for a viral post getting 50K likes per second: how do you design them?
Each like writes one (post_id, user_id) row so it can’t be duplicated and emits an event to Kafka; a counter service batches them into one HINCRBY, and periodically copies counts to the DB.
No direct UPDATE ... SET likes = likes + 1 (hot row), no COUNT(*) on read. You can also split the counter into several sub-keys.
Useful phrases for interviews
- “Before drawing anything, which posts do users actually care about?”
- “The real cost here is fan-out: ten billion inbox writes a day if we push everything.”
- “Push makes reads cheap and writes expensive; pull is the opposite.”
- “I’d go hybrid: push for regular users, pull for celebrities, and merge at read time.”
- “The feed only stores ids; we hydrate them from the post cache.”
- “We only fan out to users who’ve been active in the last seven days.”
- “A returning user has a cold start, so we build their first feed by pulling.”
- “I’d use cursor-based pagination so new posts don’t cause duplicates.”
- “Like counts are batched before they hit Redis, so a viral post doesn’t create a hot key.”
- “A few seconds of delay in the feed is acceptable, so eventual consistency is fine.”
Notes based on the System Design course (Ronin Engineer), rewritten in my own words.