1. The problem: why is ephemeral content its own design problem?

Instagram launched Stories in 2016, and within a few years publicly reported around 500 million daily Stories users. Stories look like "posts that disappear", but their access pattern is very different from feed posts:

• Everything is hot, then nothing is. A story gets almost all of its views in its first few hours and none after 24 hours. Feed posts have a long tail; stories have a cliff.
• Reads create writes. Every view is recorded (seen-state for the viewer, viewer list for the author). Stories are therefore write-heavy in a way feed browsing isn't.
• Expiry must be correct. If a story is still visible at hour 25, that's a privacy bug, not a performance issue.
• The UI demands instant tap-through. Users flick through dozens of stories in a minute; any spinner kills the experience.
• Audiences vary per story: all followers, close friends only, minus people the author hid it from.

2. Requirements and numbers

2.1 Functional requirements

2.2 Non-functional requirements

• Low latency: tray loads in <200 ms server time; next story starts in <100 ms (needs prefetch).
• High availability for viewing; brief staleness in viewer counts is fine (eventual consistency).
• Strict expiry and audience enforcement — correctness, not best effort.
• Handle extreme skew: a celebrity's story may get tens of millions of views in an hour.

2.3 Back-of-the-envelope (assumptions)

Assume 500M daily story viewers; 100M of them post, averaging 4 story items each; each viewer watches ~40 story items per day and opens the app ~10 times.

New stories/day   = 100M × 4                 = 400M   → ~4,600/s avg  (~14k/s peak)
Story views/day   = 500M × 40                = 20B    → ~230,000/s avg (~700k/s peak)
Tray loads/day    = 500M × 10                = 5B     → ~58,000/s avg
View records      = 20B × ~24 bytes (story_id, viewer_id, ts)  ≈ 480 GB/day before replication
Media ingest      = 400M × ~3 MB (photo/video mix)              ≈ 1.2 PB/day

Two striking facts: views outnumber new stories 50 : 1, and every view is a write. The design must make view-recording cheap and spread out. See capacity estimation.

3. Data model

stories              PARTITION KEY author_id   CLUSTERING story_id DESC   (TTL 24h on the "active" copy)
  author_id | story_id (time-sortable 64-bit) | media_id | media_type | audience | created_at | expires_at
            |                                  |          |            | ALL / CLOSE_FRIENDS

story_archive        PARTITION KEY author_id   CLUSTERING story_id DESC   (no TTL — private archive)
highlights           PARTITION KEY owner_id    CLUSTERING highlight_id  → title, cover_media, [story_ids...]

story_views          PARTITION KEY (story_id, bucket)  CLUSTERING viewer_id     (TTL ~48h)
story_view_counts    story_id → approximate count (counter / aggregated in a stream)

seen_watermark       (viewer_id, author_id) → last_seen_story_id   (Redis hash, TTL 24h)
tray_inbox           viewer_id → sorted set { author_id : latest_story_ts }   (Redis, entries >24h trimmed)

close_friends        PARTITION KEY owner_id  CLUSTERING friend_id
story_hidden_from    PARTITION KEY owner_id  CLUSTERING viewer_id

• Partition by author: "all active stories by author X" — the main read when you tap a tray item — is one partition scan, newest-first. This mirrors the post metadata store (Cassandra, sharded by user_id) in the diagram.
• Story IDs are time-sortable (Snowflake-style), so ordering and "newer than my watermark" comparisons need no extra timestamp index.
• Views are keyed by story, not by author, so the author can page through viewers of one story; the bucket spreads a hot story across partitions (section 7).
• Media lives in object storage, uploaded through the same pre-signed flow as posts (Media upload), and is served via the CDN.

4. Making stories expire: TTL implementations

TTL (time to live) means "this data stops existing (or stops being visible) after a set duration". There are several places to enforce it, each with different guarantees.

4.1 Why combine them

No single mechanism is both exact and complete. The robust design layers them, each doing what it's good at:

1. Correctness comes from the query-time filter on every read path (tray, author's story list, story-by-ID), plus short-lived signed media URLs so a cached URL can't be replayed for days.
2. Storage reclamation comes from Cassandra TTL (in a table dedicated to TTL'd data, with time-window compaction) and Redis EXPIRE.
3. Side effects and derived data (cache purges, tray cleanup, analytics events) come from the sweeper, driven by a delayed queue or a time-bucketed index (expiring_at_hour → [story_ids]).
4. Media is reclaimed by lifecycle rules on an ephemeral prefix — only for stories not kept in the archive or a highlight (section 9).

5. Generating the stories tray

The tray is the row of circular avatars at the top of the home screen. Building it means answering, fast: "Of the hundreds of accounts I follow, which have active stories I'm allowed to see, in what order?" This is a small version of the news feed problem, and the same three options apply.

5.1 Pull, push or hybrid

5.2 The read path

def build_tray(viewer):
    cutoff = now() - 24h
    candidates  = redis.zrangebyscore(f"tray:{viewer}", cutoff, "+inf")          # pushed authors
    candidates += [c for c in followed_celebrities(viewer)                        # pulled authors
                   if celeb_latest_story_ts(c) > cutoff]
    candidates  = [a for a in candidates if can_view_any_story(viewer, a)]        # privacy re-check
    watermarks  = redis.hmget(f"seen:{viewer}", candidates)
    for a in candidates:
        a.unseen = latest_story_id(a) > watermarks[a]
    ranked = rank_by_affinity(viewer, candidates)                                 # ML model
    return [viewer.own_story] + sorted(ranked, key=lambda a: (not a.unseen, -a.score))

Typical sizes: a viewer follows a few hundred accounts, of which maybe 30–100 have active stories — a small candidate set, so ranking it per request is affordable.

5.3 Ranking by affinity

The tray order is personalised. A ranking model (see Analytics & ML) predicts how likely the viewer is to watch and engage with each author's stories, using features such as:

• Relationship strength: DMs exchanged, likes/comments between the two, profile visits, being on each other's close friends list, mutual follows.
• Story history: how often the viewer watched this author's past stories to the end vs skipped.
• Freshness: how recently the author posted; number of unseen items.
• Content type: live videos are usually pinned near the front.

The ordering rule is roughly unseen before seen, then by predicted interest. Affinity features are precomputed offline/in streams and cached per (viewer, author) pair so scoring is cheap at request time.

6. Tracking seen-state at scale

Every view must flip a ring from coloured to grey and make playback start at the right story. At ~230k views/s, storage and write cost matter. Four designs:

6.1 Why the watermark works so well

When you tap an author, stories play chronologically, and authors only ever append new stories. So "seen" for an author collapses to one number: the highest story ID you've reached. Unseen check: author.latest_story_id > watermark. Resume point: first story with id > watermark. One small hash field per (viewer, author), updated with a "max" operation that's naturally idempotent and order-insensitive:

-- Lua in Redis: only move the watermark forward
local cur = redis.call('HGET', KEYS[1], ARGV[1])
if (not cur) or tonumber(ARGV[2]) > tonumber(cur) then
  redis.call('HSET', KEYS[1], ARGV[1], ARGV[2])
end
redis.call('EXPIRE', KEYS[1], 86400)

Trade-off: if a viewer skips ahead, skipped stories count as seen — usually acceptable product behaviour. Story IDs must be time-sortable for this to work.

Memory check: 500M viewers × ~50 authors watched × ~20 B (field + value + overhead) ≈ 500 GB — a modest sharded Redis cluster, discarded daily.

6.2 When a Bloom filter makes sense

A Bloom filter answers "definitely not seen" or "probably seen" in a tiny fixed space. With n items and m bits, the optimal false-positive rate is about (1 − e−kn/m)k; ~9.6 bits per item with 7 hash functions gives ~1%. A 1% error that marks an unseen story as seen means an author silently loses some views — bad for the main tray ring. Bloom filters fit better for "don't show me things I've probably seen" in ranking surfaces (e.g. suggested stories/Reels), where an occasional miss is harmless.

6.3 Reducing write load

• Client batching: the app marks stories seen locally (instant UI) and sends batched mark_seen calls every few seconds or when leaving the viewer — one request for 10 stories instead of 10.
• Async pipeline: the API appends view events to Kafka; consumers update watermarks, viewer lists and counts in bulk.
• Multi-device: watermarks are server-side, so seen-state syncs across phone and web.

7. Viewer lists and celebrity hot partitions

Authors can see who viewed each story. For most users that's a few dozen to a few thousand rows. For a celebrity with hundreds of millions of followers, one story may collect tens of millions of views within hours.

7.1 The hot-partition problem

If story_views is partitioned by story_id alone, all 50M view writes for one story hit one partition — owned by one small replica set in Cassandra — while the rest of the cluster idles. That node's write path, compaction and disk all saturate (a hot partition), and the partition grows to gigabytes, making reads and repairs painful. See Sharding.

 story_id only:                       (story_id, bucket = hash(viewer_id) % 64):
  node A  ████████████████████ 50M     node A ███  node B ███  node C ███  node D ███ ...
  node B  ▏                            (~780k rows per bucket, spread across the ring)
  node C  ▏

7.2 Mitigations

• Bucketed partition key: (story_id, bucket) with bucket = hash(viewer_id) % N. N can be 1 for ordinary users and larger (64–1024) for big accounts, chosen at story creation from the author's follower count. Reading the full list means querying N partitions in parallel — fine, because only the author reads it.
• Buffer and batch: view events flow through Kafka (keyed by story_id+bucket); consumers write in batches of hundreds of rows, smoothing spikes.
• Counts separately: never COUNT(*) the view list. Maintain a counter aggregated in the stream (increment by batch size), or a HyperLogLog for approximate unique viewers (~12 KB per story, ~0.8% error) — see counters at scale.
• Show a slice: the UI pages through viewers (e.g. 50 at a time) and for huge audiences shows a ranked subset (people you interact with first) plus the total.
• Short retention: viewer lists are kept only for a limited window after posting (Instagram shows viewers for up to about 48 hours), so a TTL bounds total storage.

8. Replies and reactions

Replying to a story or tapping a quick-reaction emoji doesn't create a comment; it sends a direct message to the author that references the story:

Message { thread_id, sender_id, type: "story_reply", text: "😂 so good",
          story_ref: { author_id: 7, story_id: 31415..., media_id: 991 } }

• The reply goes through the Direct Messaging service (WebSocket delivery, Messages store) and triggers a notification.
• The DM stores a reference, not a copy. When the story expires (or the viewer loses access), the thread shows "story unavailable" — permission is re-checked when rendering.
• Interactive stickers (polls, quizzes, question boxes, emoji sliders) write responses to a per-story table and maintain aggregated tallies, using the same counter techniques as likes.
• Rate limits and spam filters apply as for DMs: replies are a common spam vector.

9. Highlights and the archive

Highlights are named collections of past stories pinned to the profile, exempt from the 24-hour TTL. Instagram also keeps a private Stories Archive (on by default) so authors can revisit and re-share their old stories. So "expiry" really means public visibility expiry, not necessarily data deletion.

• Data: on creation the story is written to both the TTL'd stories table and the non-TTL story_archive (if archiving is on). A highlight stores an ordered list of archived story IDs plus a cover image.
• Media lifecycle: blobs must not be deleted by a blanket lifecycle rule if they're archived or highlighted. Options: (a) write story media to a permanent prefix when archiving is on, and to an ephemeral prefix (with a lifecycle rule) only when it's off; (b) on "add to highlight", copy the object into a permanent prefix; (c) reference-count blobs and let a sweeper delete only when the count reaches zero.
• Visibility: highlights follow the account's privacy (private account → approved followers only), and the read path still applies blocks.
• Viewer lists for highlight views are typically not retained like live story views — another reason the views table has its own TTL.

10. Prefetching for instant tap-through

Users tap through stories every 1–5 seconds. If each tap waited for a CDN fetch (50–300 ms on mobile, worse on 3G), the experience would stutter. The app therefore prefetches:

1. When the tray loads, the response already contains signed CDN URLs for each tray author's first unseen story; the app downloads the first 3–5 authors' first item in the background.
2. While you watch story k, the app fetches k+1 and k+2, plus the next author's first unseen story (because swiping to the next author is common).
3. For video, it prefetches only the first segment(s) (a second or two) of an adaptive stream, not the whole file.
4. Prefetching adapts to conditions: more on Wi-Fi, less on cellular or in data-saver mode, none on low battery; prefetches are cancelled when the user leaves.

Cost estimate: prefetching 5 photos at ~150 KB = 750 KB per tray open; at 5B tray opens/day that's ~3.75 PB/day of potentially wasted CDN egress if nobody tapped — so prefetch depth is tuned using measured tap-through probabilities. See Clients and CDN.

11. Privacy filtering

Each story has an audience rule; enforcement must be correct even as relationships change after posting.

• At fan-out time: don't push a close-friends story's author into the trays of non-close-friends; skip blocked/hidden users. This keeps trays clean and saves writes.
• At read time: re-check, because lists change after fan-out (you remove someone from close friends, block them, or they unfollow). The authoritative check runs when the tray is built and when the story list/media URLs are requested.
• Media URLs are signed and short-lived, so a URL captured before you were blocked stops working quickly.
• The close-friends list is private to its owner — the Graph service never exposes "who has me on close friends".

12. How Instagram uses all this

• Expiry: read paths filter on the story ID's embedded timestamp; TTLs and sweepers clean up; the archive and highlights retain what the author keeps.
• Celebrities: stories of very large accounts are pulled at tray-build time rather than fanned out, and their view writes are bucketed and batched.
• Experience: aggressive, adaptive prefetching makes tapping feel instant; seen-state syncs across devices via server-side watermarks.

13. Trade-offs & common pitfalls

Common pitfalls

• Relying only on TTL/lifecycle for visibility → stories visible past 24 h because compaction or lifecycle processing lags.
• Lifecycle rules deleting highlighted media → broken highlights. Keep ephemeral and permanent media separate or reference-count.
• Partitioning views by story_id alone → hot partitions for celebrity stories.
• COUNT(*) for view counts → scanning millions of rows per refresh.
• Checking privacy only at fan-out → someone removed from close friends still sees the story already in their tray.
• Using client time for expiry → users change their phone clock to see or hide stories.
• Mixing TTL'd and permanent rows in one Cassandra table → tombstone build-up and poor compaction.

14. Real-world technologies

15. Interview corner

16. Check your understanding

Related lessons

• News Feed — the same fan-out-on-write vs read trade-off, in full
• Media Upload — how story media is uploaded
• Caching — Redis data structures, EXPIRE and hot keys
• Databases — why wide-column stores suit TTL'd, append-heavy data
• Sharding — hot partitions and bucketed keys
• Likes & Counters — counting views and reactions at scale
• Social Graph — followers, close friends and blocks used for audience checks
• Direct Messaging — where story replies go
• Object Storage — lifecycle rules for media