TL;DR

1. The problem: why API design matters so much for Instagram

Every feature you see in the Instagram app — the feed, likes, comments, stories, search, DMs — is the app calling an API. Those APIs are the most permanent part of the system:

🧠 Analogy An API is a restaurant menu. Customers (clients) don't walk into the kitchen; they order from the menu using an agreed format ("one Margherita, large"). The kitchen (services) can change its chefs, ovens and suppliers freely — as long as the menu stays the same. If the restaurant suddenly renames "Margherita" to "Pizza #3", every regular customer's usual order fails. Good API design is about writing a clear menu and changing it without confusing existing customers.

2. REST principles

REST (Representational State Transfer, from Roy Fielding's 2000 dissertation) is an architectural style for APIs over HTTP. In practice, "RESTful" APIs follow these principles:

2.1 Resources (nouns), identified by URLs

Model your domain as resources — users, posts, comments, likes, stories — each with a URL. URLs are nouns; the action comes from the HTTP method.

/v1/users/42                    a user
/v1/users/42/posts              collection: posts by user 42
/v1/posts/987                   a post
/v1/posts/987/comments          collection: comments on post 987
/v1/posts/987/likes             collection: likes on post 987

Avoid verbs in paths:  POST /v1/createPost   ✖     POST /v1/posts   ✔
                       GET  /v1/getFeed      ✖     GET  /v1/feed    ✔

Conventions: plural nouns for collections, lowercase with hyphens, nesting only one or two levels deep (beyond that, use a top-level resource with a filter: /v1/comments?post_id=987). Some actions don't map neatly to CRUD — model them as sub-resources (POST /v1/users/42/follow) rather than inventing verbs.

2.2 HTTP verbs and their semantics

MethodMeaningSafe?Idempotent?Instagram example
GETRead a resource or collectionYesYesGET /v1/feed
POSTCreate a sub-resource / trigger an actionNoNo (make it so with idempotency keys)POST /v1/posts/987/comments
PUTReplace a resource entirely (create at known URL)NoYesPUT /v1/posts/987/likes/me (alternative like design)
PATCHPartially update a resourceNoNot guaranteedPATCH /v1/users/42 {"bio": "…"}
DELETERemove a resourceNoYesDELETE /v1/posts/987/likes (unlike)

Safe = doesn't change server state (so caches, prefetchers and crawlers can call it freely). Idempotent = doing it N times has the same effect as doing it once — critical because networks cause retries. Liking twice should still be one like; deleting twice should still leave it deleted.

2.3 Statelessness

Each request carries everything the server needs to process it — the access token, the cursor, the parameters. The server keeps no per-client conversation state in memory between requests. This is what lets any of thousands of stateless service instances behind the load balancer handle any request, and lets you add or remove instances freely. (Data obviously lives in databases — "stateless" refers to the server process, not the system.)

2.4 The other REST constraints, briefly

3. The Instagram API

Here is a representative public API surface for our Instagram design (an interview-style design, not Instagram's literal private API):

EndpointPurposeHandled by
POST /v1/media/upload-urlGet a pre-signed URL to upload a photo/video directly to object storageUpload Service
POST /v1/postsCreate a post from uploaded media + captionUpload Service
GET /v1/feed?cursor=&limit=Home feed pageNews Feed Service
GET /v1/users/{id}/posts?cursor=A user's profile gridPost metadata (via Upload/Post service)
GET /v1/posts/{id}One post with countsPost service
POST /v1/posts/{id}/likes / DELETELike / unlikeLike & Comment Service
GET /v1/posts/{id}/comments?cursor=List commentsLike & Comment Service
POST /v1/posts/{id}/commentsAdd a commentLike & Comment Service
POST /v1/users/{id}/follow / DELETEFollow / unfollow (or request, if private)Follow/Graph Service
GET /v1/users/{id}/followers?cursor=Follower listFollow/Graph Service
GET /v1/search?q=&type=Search users, hashtags, placesSearch Service
POST /v1/storiesPost a story (expires in 24 h)Stories Service
GET /v1/stories/trayThe row of story avatars at the top of the feedStories Service
WebSocket /v1/dmReal-time direct messages, typing indicators, read receiptsDirect Messaging
POST /v1/auth/login, /v1/auth/tokenLogin and token refreshUser/Auth Service

3.1 Uploading and creating a post (two steps)

Media bytes never flow through the API servers. The client first asks for an upload URL, uploads directly to object storage, then creates the post. See Media Upload.

POST /v1/media/upload-url
Authorization: Bearer eyJ…
Content-Type: application/json

{ "media_type": "image/jpeg", "size_bytes": 3481200, "sha256": "9f3a1c…" }

201 Created
{
  "media_id": "3209876543210987654",
  "upload_url": "https://upload.example-storage.com/raw/3209876543210987654?X-Signature=…&X-Expires=900",
  "method": "PUT",
  "expires_in": 900
}
POST /v1/posts
Idempotency-Key: 6f1c7a2e-1b9d-4c55-9b8e-2f0d7c1e4a11
{
  "media_ids": ["3209876543210987654"],
  "caption": "Sunset at Marine Drive 🌅 #mumbai",
  "location_id": "212988663",
  "user_tags": [{ "user_id": "77", "x": 0.42, "y": 0.61 }]
}

201 Created
Location: /v1/posts/3209876600000001234
{
  "id": "3209876600000001234",
  "author": { "id": "42", "username": "alice" },
  "status": "processing",          // media pipeline still creating renditions
  "created_at": "2026-10-23T09:20:11Z"
}

Note: IDs are returned as strings. Instagram-style 64-bit IDs (see ID generation) exceed JavaScript's safe integer range (253), so a JSON number would silently lose precision in the web client.

3.2 Reading the feed

GET /v1/feed?limit=10&cursor=eyJ0cyI6MTc5MzA5ODkwMCwiaWQiOiIzMjA5In0

200 OK
{
  "items": [
    {
      "id": "3209876600000001234",
      "author": { "id": "42", "username": "alice", "avatar_url": "https://cdn…/a42_150.jpg" },
      "media": [{
        "type": "image",
        "renditions": [
          { "width": 1080, "url": "https://cdn…/p/9f3a1c/1080.jpg?oh=…&oe=…" },
          { "width": 640,  "url": "https://cdn…/p/9f3a1c/640.jpg?oh=…&oe=…" }
        ]
      }],
      "caption": "Sunset at Marine Drive 🌅 #mumbai",
      "like_count": 10431,
      "comment_count": 212,
      "viewer_has_liked": false,
      "created_at": "2026-10-23T09:20:11Z"
    }
  ],
  "next_cursor": "eyJ0cyI6MTc5MzA5NTEwMCwiaWQiOiIzMTk4In0",
  "has_more": true
}

3.3 Likes, comments and follows

POST /v1/posts/3209876600000001234/likes          → 200 { "liked": true,  "like_count": 10432 }
POST /v1/posts/3209876600000001234/likes  (again) → 200 { "liked": true,  "like_count": 10432 }   # idempotent
DELETE /v1/posts/3209876600000001234/likes        → 200 { "liked": false, "like_count": 10431 }

POST /v1/posts/3209876600000001234/comments
Idempotency-Key: 0b8e…
{ "text": "Gorgeous! 😍", "reply_to_comment_id": null }
→ 201 { "id": "18000000000000077", "text": "Gorgeous! 😍", "author": {…}, "created_at": "…" }

POST /v1/users/99/follow
→ 200 { "status": "following" }          # public account
→ 200 { "status": "requested" }          # private account: pending approval

GET /v1/search?q=sunset&type=hashtag&limit=5
→ 200 { "results": [ { "type": "hashtag", "name": "sunset", "post_count": 312000000 }, … ] }

GET /v1/stories/tray
→ 200 { "tray": [ { "user": {"id":"42","username":"alice"}, "has_unseen": true,
                    "latest_reel_at": "…", "items_count": 3 }, … ] }

4. Status codes and a consistent error format

CodeMeaningWhen Instagram's API returns it
200 OKSuccess with bodyFeed page, like toggled
201 CreatedNew resource created (+ Location header)New post, new comment
202 AcceptedAccepted for async processingVideo post accepted; transcoding in progress
204 No ContentSuccess, no bodyDelete a comment
304 Not ModifiedCached version still valid (ETag)Config/feature-flag fetch unchanged
400 Bad RequestMalformed or invalid inputCaption too long, bad cursor
401 UnauthorizedNot authenticated / token expiredAccess token expired → refresh
403 ForbiddenAuthenticated but not allowedCommenting where comments are restricted
404 Not FoundDoesn't exist (or you may not know it exists)Deleted post; private post you can't see
409 ConflictState conflictUsername already taken; idempotency key reused with a different body
413 Payload Too LargeBody too bigOversized request body
422 Unprocessable EntityWell-formed but semantically invalidMedia ID not finished uploading
426 Upgrade RequiredClient version no longer supportedAncient app version → force-update screen
429 Too Many RequestsRate limitedToo many follows in an hour
500 / 502 / 503 / 504Server error / bad gateway / overloaded / upstream timeoutRetry with backoff (only if the operation is idempotent)

Rule: 4xx = the client must change something; 5xx = the server failed and a retry may succeed. Clients' retry logic depends on this distinction, so never return 200 with an error inside the body.

Every error should use one consistent shape across all services (the gateway can enforce this):

HTTP/1.1 400 Bad Request
Content-Type: application/json
X-Request-Id: 7f3c9e21-…

{
  "error": {
    "code": "CAPTION_TOO_LONG",                 // stable, machine-readable; clients switch on this
    "message": "Captions can be at most 2,200 characters.",   // human-readable, localisable
    "details": [ { "field": "caption", "issue": "max_length", "limit": 2200 } ],
    "request_id": "7f3c9e21-…",                 // lets support/engineers find the trace
    "retryable": false,
    "doc_url": "/docs/errors#CAPTION_TOO_LONG"
  }
}

Clients should switch on the stable code, never on the English message. RFC 9457 "Problem Details for HTTP APIs" (application/problem+json) standardises a similar format with type, title, status, detail. Don't leak internals (stack traces, SQL) in errors.

5. Pagination in depth

No endpoint should return an unbounded list — a celebrity has hundreds of millions of followers; a viral post has millions of comments. Lists are returned a page at a time.

5.1 Offset pagination

GET /v1/users/42/posts?limit=12&offset=24          (page 3)

SELECT id, caption, created_at
FROM posts
WHERE author_id = 42
ORDER BY created_at DESC, id DESC
LIMIT 12 OFFSET 24;

Simple, supports "jump to page 50", and is fine for small, stable lists. But it has two serious problems:

  1. Performance degrades with depth. OFFSET 100000 means the database reads and throws away 100,000 rows before returning 12. Deep pages get linearly slower (O(offset + limit)).
  2. Items shift when data changes. If a new post is inserted at the top while you're paging, everything shifts down by one: the first item of page 2 is the last item you already saw on page 1 (duplicate). If an item is deleted, you skip one (missing item). On a feed where new content arrives every second, this happens constantly.
 Page 1 fetched (offset 0, limit 3):     [P10, P9, P8]
 New post P11 arrives at the top.        list is now [P11, P10, P9, P8, P7, P6 ...]
 Page 2 fetched (offset 3, limit 3):     [P8, P7, P6]   ← P8 shown TWICE

5.2 Cursor / keyset pagination

Instead of "skip N rows", the client says "give me items after this one". The server returns an opaque cursor encoding the sort key of the last item; the next query uses it in a WHERE clause that an index can jump to directly.

-- first page
SELECT id, caption, created_at
FROM posts
WHERE author_id = 42
ORDER BY created_at DESC, id DESC
LIMIT 12;

-- next page: last item seen had created_at = '2026-10-20 18:03:11', id = 3198000000000000999
SELECT id, caption, created_at
FROM posts
WHERE author_id = 42
  AND (created_at, id) < ('2026-10-20 18:03:11', 3198000000000000999)   -- row-value comparison
ORDER BY created_at DESC, id DESC
LIMIT 12;

-- supporting index
CREATE INDEX idx_posts_author_time ON posts (author_id, created_at DESC, id DESC);
OFFSET 100000 LIMIT 12 WHERE id < cursor LIMIT 12 100,000 rows read and thrown away 12 rows returned B-tree index (author_id, id) seek directly 12 rows read, 12 returned cost grows with page depth: O(offset + limit) constant cost per page: O(log n + limit)
Offset pagination scans and discards rows; keyset pagination seeks straight to the right place in the index.

5.3 Offset vs cursor compared

Offset / page numberCursor / keyset
Deep-page performanceDegrades linearlyConstant (index seek)
Stable under inserts/deletesNo — duplicates and gapsYes — anchored to the last item seen
Jump to arbitrary pageYesNo (only next/previous)
Total count / "page 7 of 40"Natural (but COUNT(*) is expensive at scale)Not natural
Works across sharded / non-SQL storesPoorly (global offsets across shards are expensive)Well (each shard can seek by key)
Best forAdmin tables, small result sets, search result pagesFeeds, timelines, comments, followers, infinite scroll
💡 Why cursor wins for feeds A feed is an infinite scroll over a list where new items constantly arrive at the top — exactly where offsets produce duplicates — and users never "jump to page 37". Ranked feeds add a twist: the ranking can change between requests, so the Feed service often snapshots the ranked candidate list for a session in cache and the cursor encodes a position in that snapshot (plus the IDs already served), guaranteeing no repeats.

6. Idempotency keys

A user taps "Post comment" on a train. The request reaches the server and the comment is saved, but the response is lost in a tunnel. The app times out and retries. Without protection, the comment is posted twice. GET, PUT and DELETE are idempotent by design; POST is not. The fix is an idempotency key: a unique ID (UUID v4) the client generates once per user action and sends with every retry of that action.

 App                                   Comment Service                    dedupe table
  |-- POST /comments  Key: K1 ------->  | INSERT key K1 (in_progress) --->| ok (new)
  |                                     | create comment C77               |
  |                                     | UPDATE K1 → done, response=201 C77
  |   ✖ response lost (timeout)         |                                  |
  |-- retry POST /comments Key: K1 ---> | INSERT K1 → conflict, exists --->| status=done
  |<-- 201 C77 (stored response replayed, no second comment) ------------|

6.1 Implementing it with a dedupe table

CREATE TABLE idempotency_keys (
  user_id        BIGINT       NOT NULL,
  idem_key       UUID         NOT NULL,
  request_hash   BYTEA        NOT NULL,     -- hash of method+path+body, to detect key reuse with different payload
  status         VARCHAR(12)  NOT NULL,     -- in_progress | completed
  response_code  SMALLINT,
  response_body  JSONB,
  created_at     TIMESTAMPTZ  NOT NULL DEFAULT now(),
  PRIMARY KEY (user_id, idem_key)           -- scoped per user so keys can't collide across users
);
-- expire rows after e.g. 24 h with a TTL job (or use Redis SET NX EX / Cassandra TTL)
def handle_create_comment(user_id, key, request):
    h = sha256(request.method + request.path + request.body)
    inserted = db.execute("""INSERT INTO idempotency_keys (user_id, idem_key, request_hash, status)
                             VALUES (%s, %s, %s, 'in_progress')
                             ON CONFLICT DO NOTHING""", user_id, key, h).rowcount == 1
    if not inserted:
        row = db.get_key(user_id, key)
        if row.request_hash != h:      return 422/409 "Idempotency key reused with different request"
        if row.status == "in_progress": return 409 "Request in progress, retry shortly"
        return Response(row.response_code, row.response_body)      # replay original result

    try:
        with db.transaction():                                   # ideally same DB/transaction as the write
            comment = comments.insert(...)
            db.update_key(user_id, key, status="completed", code=201, body=serialize(comment))
        return Response(201, comment)
    except RetryableError:
        db.delete_key(user_id, key)     # let the client retry for real
        raise

7. Versioning strategies

Eventually you'll need a breaking change: remove a field, change a type, change semantics. Versioning lets old clients keep the old contract while new clients get the new one.

StrategyExampleProsCons
URI path/v1/feed, /v2/feedExplicit, visible in logs, easy routing at the gateway, easy to cacheCoarse: whole API bumps; URLs aren't "pure" resource IDs
Query parameter/feed?api_version=2Easy to addEasy to forget; messy caching
HeaderAccept: application/vnd.instagram.v2+json or Api-Version: 2026-10-01Clean URLs; fine-grainedLess visible; harder to test in a browser
Date-based versionsStripe-Version: 2024-06-20Each account pinned to a version; small incremental changesServer keeps transformation layers for every date
No versions, additive-only evolutionAdd fields; never remove/rename; GraphQL deprecationsNo version explosionRequires strict discipline; cruft accumulates

A practical combo: a major version in the path (/v1) changed very rarely, plus additive, backward-compatible evolution within it, plus the client app version sent in a header (User-Agent / X-App-Version: 312.0.0) so the server can adapt behaviour or responses for specific old builds.

8. Filtering, sorting, field selection and partial responses

GET /v1/users/42/posts?media_type=video&since=2026-01-01&limit=12         # filtering
GET /v1/posts/987/comments?sort=top                                      # sorting (top vs newest)
GET /v1/users/42?fields=id,username,avatar_url,follower_count            # field selection (sparse fieldsets)
GET /v1/posts/987?expand=author,top_comments                              # embed related resources

9. gRPC: fast internal service-to-service calls

gRPC (open-sourced by Google in 2015) is an RPC framework: you call a method on a remote service as if it were a local function. Its building blocks:

syntax = "proto3";
package graph.v1;

service GraphService {
  rpc GetFollowers (GetFollowersRequest) returns (GetFollowersResponse);              // unary
  rpc CheckRelationships (CheckRelationshipsRequest) returns (CheckRelationshipsResponse);
  rpc StreamFollowers (GetFollowersRequest) returns (stream User);                    // server streaming
}

message GetFollowersRequest {
  int64  user_id = 1;
  string cursor  = 2;
  int32  limit   = 3;
}
message User { int64 id = 1; string username = 2; string avatar_url = 3; }
message GetFollowersResponse { repeated User users = 1; string next_cursor = 2; }

message CheckRelationshipsRequest { int64 viewer_id = 1; repeated int64 target_ids = 2; }
message CheckRelationshipsResponse { map<int64, Relationship> relationships = 1; }
message Relationship { bool following = 1; bool blocked_either_way = 2; bool close_friend = 3; }

Why it's typical internally: the Feed service may call the Graph service, Post service and ranking service dozens of times per feed request; binary encoding, persistent multiplexed connections and generated typed clients save CPU and latency at that volume. Why it's less common for public/browser clients: browsers can't speak native gRPC (they need gRPC-Web via a proxy), binary payloads are harder to debug, and HTTP caching/CDNs don't apply naturally. The API gateway often translates external JSON/REST into internal gRPC. Meta internally uses its own Thrift-based RPC framework — the same idea (IDL + binary encoding + generated clients); Thrift itself was created at Facebook.

10. GraphQL

GraphQL was created at Facebook in 2012 to power its mobile News Feed and open-sourced in 2015. It's a query language plus runtime: the server publishes a typed schema; the client sends a query naming exactly the fields it needs, usually to a single endpoint (POST /graphql).

10.1 Over-fetching and under-fetching

query PostDetail($id: ID!) {
  post(id: $id) {
    id
    caption
    likeCount
    viewerHasLiked
    author { id username avatarUrl(size: 150) viewerIsFollowing }
    comments(first: 3) {
      edges { node { id text author { username } } }
      pageInfo { endCursor hasNextPage }       # cursor pagination built into the Relay spec
    }
  }
}

mutation Like($id: ID!) { likePost(postId: $id) { post { id likeCount viewerHasLiked } } }

Also: mutations for writes, subscriptions for real-time updates (typically over WebSockets), introspection for tooling, and the Relay connection spec standardising cursor pagination.

10.2 GraphQL challenges

11. REST vs gRPC vs GraphQL

REST (JSON over HTTP)gRPCGraphQL
ModelResources + HTTP verbsRemote procedure calls (methods)Typed graph; client-specified queries
PayloadJSON (text)Protobuf (binary, compact)JSON response shaped like the query
TransportHTTP/1.1, 2 or 3HTTP/2 (required)Usually HTTP POST; WebSocket for subscriptions
ContractOptional (OpenAPI)Mandatory .proto, code-generatedMandatory schema, introspectable
Fetching efficiencyOver/under-fetching commonFixed messages per methodExactly the fields requested
StreamingLimited (SSE, chunked)Native: client, server, bidirectionalSubscriptions
HTTP caching / CDNExcellent (GET + Cache-Control/ETag)PoorPoor unless persisted GET queries
Browser supportNativeNeeds gRPC-Web proxyNative
DebuggabilityEasy (curl, readable)Needs tooling (grpcurl)Good tooling (GraphiQL)
Best forPublic APIs, simple CRUD, cacheable readsInternal microservice calls, low latency, polyglotClient-driven UIs with many screens and nested data
In our Instagram designPublic mobile/web API (this lesson's table)Feed ↔ Graph ↔ Post ↔ Ranking servicesGateway/BFF layer for complex screens

12. Real-time: polling, long polling, SSE and WebSockets

HTTP is request/response: the client asks, the server answers. But DMs, typing indicators, "X is live" banners and new-comment counts need the server to push. Four techniques:

Short pollingLong pollingSSEWebSocket clientserver clientserver clientserver clientserver "nothing new" "nothing new" server holds request open… event! one HTTP response, events streamed upgrade, then both directions anytime
Blue = client → server, green = server → client. Polling wastes requests; long polling waits; SSE streams one way; WebSockets are full duplex.

12.1 Short polling

The client asks every N seconds: GET /v1/dm/threads/55/messages?after=msg_900. Trivial to build and works everywhere, but most responses are empty and latency averages N/2. Worked example: 50M online users polling every 5 s = 10M requests/s, nearly all returning "nothing new" — and still up to 5 s delay. Fine for rarely-changing data (badge counts refreshed every minute).

12.2 Long polling

The client sends a request; the server holds it open until there's new data (or ~30 s timeout), then responds; the client immediately re-requests. Near-instant delivery with plain HTTP, but each event costs a full request/response cycle, and the server must hold many idle connections. Was common before WebSockets were widely supported (early Facebook chat used it).

12.3 Server-Sent Events (SSE)

The client opens GET /v1/live/events with Accept: text/event-stream; the server keeps the response open and writes events as lines of text (data: {...}\n\n). Built into browsers (EventSource) with automatic reconnection and Last-Event-ID resume. One-way (server → client) — the client sends data via normal HTTP requests. Great for live like counters, notifications, live-stream comment feeds.

12.4 WebSockets

The client sends an HTTP request with Upgrade: websocket; after the 101 Switching Protocols handshake, the TCP connection becomes a persistent, full-duplex channel carrying small frames in both directions (2–14 bytes of framing overhead per message). Ideal for chat, typing indicators, presence and multiplayer.

// client → server over WebSocket /v1/dm
{ "type": "send_message", "client_msg_id": "c-81f2", "thread_id": "55", "text": "On my way!" }
// server → client (ack, carries server-assigned ID and ordering)
{ "type": "ack", "client_msg_id": "c-81f2", "message_id": "3209900000000000777", "ts": 1793099901 }
// server → recipient
{ "type": "new_message", "thread_id": "55", "message_id": "3209900000000000777", "from": "42", "text": "On my way!" }
{ "type": "typing", "thread_id": "55", "user_id": "42", "state": "start" }

Costs: stateful connections (a DM server must track which user is connected to which machine — see Direct Messaging), load balancers must support long-lived connections, deploys must drain connections gracefully, and mobile OSes kill background sockets (so offline users get push notifications via APNs/FCM instead). Client-generated client_msg_ids make sends idempotent across reconnects.

12.5 Comparison

Short pollingLong pollingSSEWebSockets
DirectionClient pullsClient pulls (server delays answer)Server → clientBoth ways
LatencyUp to the poll intervalNear real-timeReal-timeReal-time
OverheadHigh (many empty requests)Medium (request per event)LowLowest per message
Server stateNoneHeld requestsOpen connectionsOpen connections + session
Infra friendlinessWorks everywhereWorks everywhere; timeouts to tunePlain HTTP; HTTP/2 friendlyNeeds LB/proxy support for upgrades and long-lived connections
Instagram useInfrequent refresh (e.g. config)Fallback when sockets blockedLive counters / notifications on webDMs, typing, presence, live interactions

Note: Facebook Messenger and Instagram's mobile messaging have publicly been described as using MQTT (a lightweight publish/subscribe protocol designed for unreliable networks) over persistent connections — the same "persistent bidirectional connection" idea as WebSockets, optimised for battery and bandwidth. Our diagram labels it simply as WebSocket.

13. Backward compatibility for old mobile clients

Assume ~10% of active users run an app version older than 6 months, and some phones never update. Rules that keep them working:

➕ Additive changes only

Add new optional fields, endpoints and enum values. Never remove or rename fields or change their types/meaning within a version.

🧭 Tolerant reader

Clients must ignore unknown fields and handle unknown enum values with a sensible default (e.g. unknown media type → "update the app to view").

🔢 Send the app version

Every request carries X-App-Version; the server/BFF can tailor responses (e.g. don't send Reels to builds that can't render them).

🚩 Feature flags

Server-driven flags turn features on per version/user, decoupling releases from deployments.

📉 Deprecation telemetry

Measure traffic per endpoint per app version; only remove an old path when its usage is negligible.

⛔ Minimum supported version

For truly unsupported builds (security fixes), return 426 / a force-upgrade payload that old apps were designed to display.

Contract tests (e.g. golden JSON responses per supported app version, protobuf breaking-change detectors like buf breaking) catch accidental breaks in CI. With protobuf: never reuse a field number, mark removed fields reserved. With GraphQL: mark fields @deprecated and track usage before removal.

14. API security basics

15. How Instagram uses these API styles

📸 In Instagram Instagram's apps talk to a private HTTP/JSON API (historically served by its Django/Python backend under paths like /api/v1/…) and, increasingly, GraphQL — which Meta created for exactly this mobile use case. Internally, Meta services communicate over Thrift-based RPC, and Instagram/Messenger messaging has been publicly described as using MQTT over persistent connections. Our course design maps these onto: REST/JSON (and GraphQL) at the edge, gRPC-style RPC between services, WebSockets for DMs.
📱 AppREST/JSON or GraphQL
→HTTPS
API Gateway / BFFauth, version, transcoding
→gRPC
Feed Service
→gRPC (batch)
Graph / Post / Ranking
→
Redis · Cassandra · PostgreSQL

15.1 Opening the app: one feed request

  1. App sends GET /v1/feed?limit=10 with Authorization, X-App-Version, Accept-Encoding: br.
  2. Gateway authenticates, rate limits, routes to the Feed service (translating JSON ↔ gRPC if needed).
  3. Feed service reads the user's precomputed timeline from Redis (post IDs), calls the ranking service, then hydrates posts with batched gRPC calls: Post.GetPosts(ids), Graph.CheckRelationships(viewer, authors), Likes.GetCounts(ids).
  4. It filters anything the viewer can't see, builds signed CDN URLs, and returns 10 items with a next_cursor that encodes the position in the ranked snapshot.
  5. The app renders, fetches images from the CDN, and as the user nears the end, requests the next page with the cursor. A like taps POST /v1/posts/{id}/likes — naturally idempotent.

15.2 Rough numbers: payload size matters

Assume 500M DAU × 30 feed pages/day = 15B feed responses/day.
JSON page with full objects: ~40 KB; with field selection + brotli: ~8 KB.
Saving: 32 KB × 15B ≈ 480 TB/day of mobile egress (≈ 44 Gbps average) — for one endpoint.
Internal: one feed page → ~5 batched RPCs; protobuf vs JSON cuts serialization CPU by several ×
across ~75B internal calls/day.

16. Trade-offs & common pitfalls

✅ Good API design gives you

  • Clients and services that evolve independently
  • Safe retries on flaky mobile networks (idempotency)
  • Stable, fast pagination over huge, changing lists
  • Smaller payloads, less battery and data use
  • Fewer security bugs through consistent conventions

⚠️ Tensions to manage

  • Flexibility (GraphQL) vs cacheability and cost control (REST)
  • Efficiency (gRPC binary) vs debuggability and browser support
  • Supporting old clients vs accumulating compatibility cruft
  • Real-time push vs stateful connection infrastructure
  • Cursor pagination's speed vs no random page access

Common pitfalls

17. Real-world technologies

TechnologyWhat it's forWhen to use
OpenAPI (Swagger)Describe REST APIs; generate docs, clients, validatorsAny REST API with multiple consumers
gRPC + Protocol Buffers, bufRPC framework, schema, breaking-change checksInternal service-to-service calls
Apache ThriftRPC/IDL framework created at FacebookSimilar to gRPC; common in older large-scale stacks
GraphQL (Relay, Apollo, graphql-java, Hot Chocolate)Client-driven queries, federationProduct UIs with many screens and nested data
WebSocket servers (Socket.IO, Netty, uWebSockets), MQTT brokersPersistent bidirectional connectionsChat, presence, live updates
SSE (EventSource)Simple server push over HTTPOne-way live feeds and notifications on web
Postman, curl, grpcurl, GraphiQLTesting and explorationDevelopment and debugging

18. Interview corner

Design the core APIs for Instagram.

Resources: users, posts, media, comments, likes, follows, stories, feed, search, DMs. Endpoints: POST /v1/media/upload-url (pre-signed upload), POST /v1/posts (with Idempotency-Key; returns 201/202 while processing), GET /v1/feed?cursor&limit, GET /v1/users/{id}/posts?cursor, POST|DELETE /v1/posts/{id}/likes, GET|POST /v1/posts/{id}/comments, POST|DELETE /v1/users/{id}/follow, GET /v1/search?q&type, GET /v1/stories/tray, WebSocket /v1/dm. Cross-cutting: Bearer tokens, cursor pagination everywhere, string IDs, a consistent error envelope, 429 with Retry-After, path versioning with additive evolution, signed CDN URLs in responses rather than media bytes.

Offset or cursor pagination for the feed? Explain with SQL.

Cursor/keyset. Offset (LIMIT 10 OFFSET 1000) scans and discards rows, so deep pages get slower, and inserts at the top shift items causing duplicates/gaps during infinite scroll. Keyset uses the last seen sort key: WHERE (created_at, id) < (:ts, :id) ORDER BY created_at DESC, id DESC LIMIT 10 with an index on (author_id, created_at, id) — an index seek, constant cost, stable under inserts. Return an opaque, signed cursor. For a ranked feed, the cursor encodes a position in a cached ranked snapshot.

How do you make "create post" safe to retry?

Client generates an Idempotency-Key (UUID) per user action and resends it on every retry. Server inserts (user_id, key, request_hash, in_progress) into a dedupe table with a unique constraint; if it already exists and is completed, replay the stored response; if in progress, return 409; if the request hash differs, reject. Perform the post creation and mark the key completed atomically (same transaction, or deterministic post IDs derived from the key so duplicate writes are upserts). Expire keys after ~24 h.

When would you use gRPC vs REST vs GraphQL in this system?

REST/JSON for the public mobile/web API — universally supported, cacheable, easy to debug, natural with a gateway. gRPC for internal service-to-service calls: compact protobuf, HTTP/2 multiplexing, generated typed clients, deadlines and streaming — matters when one feed request makes dozens of internal calls. GraphQL at the edge/BFF for complex client screens to avoid over/under-fetching and multiple round trips, with persisted queries and cost limits.

How would you deliver DMs in real time? Why not polling?

Persistent bidirectional connections (WebSockets, or MQTT on mobile) from the app to a gateway tier of connection servers; a session registry maps user → connection server; messages are persisted to the message store, then routed to the recipient's connection; offline recipients get push notifications. Polling at 50M concurrent users every 5 s would be 10M req/s of mostly empty responses with seconds of latency; long polling helps latency but still costs a request per event. SSE is one-way, and chat needs client → server typing/read events too.

You need to rename a field in the feed response. Millions of users run old app versions. What do you do?

Don't rename in place. Add the new field alongside the old one (populate both), ship app versions that read the new field, monitor usage of the old field by app version, and remove it only when traffic from versions that need it is negligible — or in the next major API version. Alternatively, have the mobile BFF shape responses per X-App-Version. Add contract tests to prevent accidental breaks.

19. Check your understanding

Which of these are idempotent: (a) GET /v1/feed, (b) POST /v1/posts/9/comments, (c) DELETE /v1/posts/9/likes, (d) PUT /v1/users/42/settings?

(a), (c) and (d). Repeating them leaves the server in the same state. (b) creates a new comment each time unless you add an idempotency key.

Offset pagination: the query LIMIT 20 OFFSET 200000 takes 900 ms while page 1 takes 3 ms. Why, and what's the fix?

The database must walk and discard 200,000 rows before returning 20 — cost grows with the offset. Switch to keyset pagination (WHERE id < :last_id ORDER BY id DESC LIMIT 20) backed by an index; every page then costs about the same as page 1.

A post ID is 3209876600000001234. Why return it as a JSON string?

JavaScript numbers are IEEE-754 doubles, exact only up to 253 − 1 ≈ 9.007 × 1015. This ID (~3.2 × 1018) would be rounded, so the web client would request the wrong post. Strings preserve every digit.

The client gets a 503 on POST /v1/posts with no idempotency key. Should it retry automatically?

Not safely. A 503 (or a timeout) doesn't tell you whether the post was created before the failure. Retrying could create a duplicate. With an idempotency key, retrying is safe — the server replays the original result if it already succeeded.

40M users are online; you consider short polling every 10 s for new-notification badges. How many requests/s is that, and what alternatives exist?

40M / 10 s = 4M requests/s, mostly empty. Alternatives: piggyback badge counts on other responses, use a push channel (the existing DM/MQTT/WebSocket connection, or SSE on web), or poll much less often with push notifications for urgent events.

Which real-time technique fits each: (a) live like counter on a web post page, (b) DMs with typing indicators, (c) refreshing remote config once an hour?

(a) SSE — one-way server push, simple over HTTP. (b) WebSockets (or MQTT) — bidirectional, low-latency. (c) Short polling — infrequent and simple; push infrastructure isn't worth it.

A client sends PATCH /v1/users/42 {"bio": "hi", "is_verified": true} and becomes verified. Name the vulnerability and the fix.

Mass assignment. The handler blindly copied request fields onto the model. Fix: whitelist client-writable fields (bio, display name, etc.) per endpoint and ignore or reject anything else; enforce authorization on sensitive fields server-side.

🎯 Key takeaways

Related lessons