1. The problem: why does Instagram need an API gateway?

Look at the main architecture diagram. Behind the load balancer there are at least eight application services: Upload, Stories, Like & Comment, Follow/Graph, News Feed, Search, Direct Messaging and User/Auth. In a real company there are hundreds. Now imagine the mobile app talking to each of them directly:

• The client would need to know every service's address. If the Feed team splits their service into two, every installed copy of the app (some of which are years old and never updated) breaks.
• Every service would re-implement the same plumbing. Validating the login token, checking rate limits, terminating TLS, writing access logs, emitting metrics, adding trace IDs, handling CORS for the web client. Eight teams writing the same security-critical code eight slightly different ways is a recipe for bugs — one forgotten auth check and private photos leak.
• The attack surface would be huge. Every service would be exposed to the public internet and need its own hardening.
• Mobile networks are slow. Rendering one profile screen needs data from the User service, the Graph service (follower counts), the Post service (grid) and the Stories service (highlights). Four separate round trips over a 4G connection with 100 ms latency each is painful.

The fix is to put one well-defined front door in front of all the services. Clients talk to i.instagram.com (one host, one API surface); the gateway figures out who you are, whether you're allowed in, whether you're sending too much, and which internal service should handle your request.

2. Reverse proxy vs load balancer vs API gateway

These three terms overlap and are often used loosely — the same piece of software (NGINX, Envoy) can play all three roles. It helps to define them by what they know.

2.1 Forward proxy vs reverse proxy

A proxy is a server that forwards traffic on someone else's behalf.

• A forward proxy acts for clients: a corporate proxy that all employees' browsers go through. The destination website sees the proxy, not the employee.
• A reverse proxy acts for servers: clients connect to it believing it is "the website", and it forwards requests to one or more hidden backend servers. The client never sees the backends' addresses. Typical jobs: TLS termination, compression, static file serving, caching, buffering slow clients, and hiding the internal topology.

2.2 The three roles compared

A useful mental model: every API gateway is a reverse proxy; every L7 load balancer is a reverse proxy; an API gateway usually contains a load balancer (once it decides "this goes to the Feed service", it still has to pick one of the 2,000 Feed service instances). See Load Balancing for the balancing algorithms.

3. Gateway responsibilities, one by one

Each responsibility below is a filter (also called a plugin, policy or middleware) in a pipeline. A request passes through them in order; any filter can short-circuit the pipeline by returning an error (e.g. 401, 429) without ever touching a backend service.

  request ─▶ [TLS] ─▶ [CORS] ─▶ [AuthN] ─▶ [Rate limit] ─▶ [Validate] ─▶ [Transform] ─▶ [Route/LB] ─▶ service
                                   │            │               │
                                  401          429             400        (short-circuit, backend never sees it)
  response ◀─ [Log/metrics/trace] ◀─ [Cache store] ◀─ [Transform response] ◀─ [Aggregate] ◀──────────── service

3.1 Routing

The gateway maps an incoming request to an upstream service using rules on the host, path, method and sometimes headers:

routes:
  - match: { method: POST,   path: "/v1/media/upload-url" }     -> upload-service
  - match: { method: POST,   path: "/v1/posts" }                -> upload-service
  - match: { method: GET,    path: "/v1/feed" }                 -> feed-service
  - match: { path_prefix: "/v1/posts/{id}/likes" }              -> like-comment-service
  - match: { path_prefix: "/v1/posts/{id}/comments" }           -> like-comment-service
  - match: { path_prefix: "/v1/users/{id}/follow" }             -> graph-service
  - match: { method: GET,    path: "/v1/search" }               -> search-service
  - match: { path_prefix: "/v1/stories" }                       -> stories-service
  - match: { path: "/v1/dm", header: "Upgrade: websocket" }     -> dm-service  (long-lived)
  - match: { path_prefix: "/v1/auth" }                          -> auth-service

After picking the service, the gateway finds healthy instances through service discovery (a registry like Consul, ZooKeeper, Kubernetes endpoints or a DNS name) and picks one using a load-balancing algorithm. Because services can be moved, split or renamed by changing this table, the public API is decoupled from the internal topology — the single biggest architectural benefit.

3.2 Authentication (token validation)

Authentication answers "who is this?". After login, the app holds an access token (often a JWT — see Authentication & Authorization). On every request the gateway:

1. Extracts the token from Authorization: Bearer <token>.
2. Validates it: for a signed JWT, it checks the signature using the auth service's public key (cached locally — no network call), the expiry (exp), the issuer and audience. For an opaque token, it looks it up in a fast session store (e.g. Redis) or calls the auth service's introspection endpoint, usually with a short local cache.
3. Rejects invalid/expired tokens with 401 Unauthorized.
4. Strips the raw token (optional) and forwards trusted identity headers such as X-User-Id: 1234, X-Device-Id, X-Auth-Scopes to the backend, ideally inside an internal signed envelope so services can trust them.

3.3 Rate limiting

The gateway is the natural place to enforce limits because it sees every request and already knows the caller's identity. Typical policies: "100 requests/second per user", "60 follow actions per hour per account", "5 login attempts per minute per IP", "5,000 calls/hour per third-party API key". Over the limit → 429 Too Many Requests with a Retry-After header. Counters usually live in a shared Redis cluster so all gateway instances agree. The algorithms (token bucket, sliding window…) are covered in depth in Rate Limiting.

3.4 TLS termination

TLS termination means the gateway (or the LB in front of it) performs the HTTPS handshake and decrypts traffic, so backend services receive plain HTTP or re-encrypted internal mTLS. Benefits:

• Certificates and private keys live in one hardened place instead of on hundreds of services.
• The expensive asymmetric crypto of the handshake is done once, on machines tuned for it (session resumption, TLS 1.3, hardware acceleration).
• The gateway can see the HTTP request — it must, to route by path or validate tokens.

Options: terminate at the edge (simplest), terminate and re-encrypt to backends (zero-trust networks — usually mTLS via a service mesh), or TLS passthrough (an L4 LB forwards encrypted bytes; the gateway terminates). See Networking Basics for how the handshake works.

3.5 Request validation

Reject malformed requests before they consume backend resources: required fields present, types correct, string lengths within limits (caption ≤ 2,200 characters, comment ≤ 300 characters — assume these limits), body size ≤ a maximum, content type allowed, no unknown enum values. Usually driven by a schema (OpenAPI/JSON Schema, or protobuf definitions). Failures return 400 Bad Request or 413 Payload Too Large. This also blunts some attacks (giant payloads, deeply nested JSON bombs).

3.6 Transformation (request and response)

The gateway can rewrite traffic so clients and services evolve independently:

• Protocol translation: client speaks HTTP/JSON, internal services speak gRPC/protobuf. The gateway converts (e.g. Envoy's gRPC-JSON transcoder).
• Path rewriting: /v1/users/42/posts → /internal/posts?author_id=42.
• Header manipulation: add X-Request-Id, X-User-Id; remove internal headers from responses (Server, debug headers) so internals don't leak.
• Version adaptation: an old app sends the v1 shape; the gateway maps it to the v2 internal shape. Useful when you cannot force-upgrade millions of installed apps.
• Response shaping: strip fields a given client does not need, rename fields, compress (gzip/brotli).

3.7 Response aggregation / composition

One client request fans out to several services in parallel, and the gateway merges the results. This turns N round trips over a slow mobile network into one, and the fan-out happens over the fast data-center network (sub-millisecond hops).

 App ── GET /v1/profile/natgeo ──▶ Gateway / BFF
                                     ├──▶ User service      (name, bio, avatar)         12 ms
                                     ├──▶ Graph service     (followers, following)       8 ms
                                     ├──▶ Post service      (first 12 grid thumbnails)  20 ms
                                     └──▶ Stories service   (highlights, active story)  10 ms
                                     (all in parallel → total ≈ max = 20 ms + merge)
 App ◀── one JSON document ──────────┘

Worked example: on a 4G connection with an 80 ms round-trip time, four sequential client calls cost ≈ 4 × 80 = 320 ms of network alone. With aggregation: one 80 ms round trip + ~20 ms of parallel internal calls ≈ 100 ms. That's a 3× faster profile screen.

Aggregation needs care: set a timeout per dependency and decide what's optional. If the Stories service is slow, return the profile without highlights (a partial response) rather than failing the whole page — see Resilience patterns. Heavy aggregation logic is better placed in a BFF or GraphQL layer (below) than in a generic gateway.

3.8 Caching

A gateway can cache responses for public, non-personalised GET requests: e.g. a hashtag's top posts for 30 seconds, public profile metadata for 10 seconds, config/feature-flag payloads for a minute. Cache keys must include anything that changes the response (path, query, Accept-Language, API version, and the user ID if the response is personalised — in which case shared caching rarely pays off). Personalised data like the feed is cached deeper, in the Redis feed cache owned by the Feed service. Media bytes are cached by the CDN, not the gateway.

3.9 CORS

CORS (Cross-Origin Resource Sharing) is the browser mechanism that decides whether JavaScript on one origin (say https://www.instagram.com) may call an API on another origin (https://i.instagram.com). For "non-simple" requests (e.g. with an Authorization header or JSON body), the browser first sends an OPTIONS preflight request. The gateway answers it centrally:

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://www.instagram.com
Access-Control-Allow-Methods: GET, POST, DELETE
Access-Control-Allow-Headers: Authorization, Content-Type, Idempotency-Key
Access-Control-Allow-Credentials: true
Access-Control-Max-Age: 600

Handling CORS at the gateway means one consistent allow-list, and preflights never reach backend services. Never reflect arbitrary Origin values together with Allow-Credentials: true — that effectively disables the browser's protection. CORS only matters for browsers; native mobile apps are not subject to it.

3.10 Logging, metrics and tracing

Because every request passes through it, the gateway is the best place to observe the whole API (see Observability):

• Access logs: timestamp, request ID, user ID, route, status, latency, bytes, client version, region. (Never log tokens or passwords.)
• Metrics: requests/sec, error rate and p50/p99 latency per route and per upstream service — the "RED" metrics (Rate, Errors, Duration). These drive dashboards and alerts.
• Distributed tracing: the gateway generates a trace ID (W3C traceparent header) and propagates it; every downstream service adds spans, so one request can be followed across 20 services.

3.11 Canary releases and A/B routing

Since the gateway decides where requests go, it can split traffic deliberately:

• Canary release: deploy feed-service v2 to a few instances and route 1% of traffic to it. Watch error rate and latency; ramp 1% → 5% → 25% → 100%, or roll back instantly by changing the weight. Bad releases hurt 1% of users for minutes instead of everyone.
• Sticky splitting: choose the bucket by hashing the user ID — hash(user_id) % 100 < 5 — so the same user consistently sees the same version (no flip-flopping between requests).
• A/B experiments: route users in the experiment group to a new ranking model or a new endpoint implementation; metrics are tagged with the variant. (Many product experiments are instead decided inside services by an experimentation framework; the gateway handles infrastructure-level splits.)
• Header-based routing: internal employees (X-Dogfood: 1) or a specific app build go to a staging backend.
• Shadow traffic (mirroring): copy live requests to a new service version and discard its responses — test at production load with zero user impact.

route: GET /v1/feed
  split:
    - weight: 99   upstream: feed-service-stable
    - weight: 1    upstream: feed-service-canary      # sticky on hash(user_id)
  mirror: feed-service-shadow (10% of requests, responses ignored)

3.12 Other responsibilities you'll hear about

4. The Backend-for-Frontend (BFF) pattern

A single generic gateway serving every client eventually becomes awkward. The iOS app, the Android app and the web app need different shapes of data:

• Mobile wants compact payloads (metered data, slow networks), smaller image renditions, and aggregated screens ("everything for the feed screen in one call").
• Web wants larger renditions, SEO-friendly server-side rendering data, and can tolerate more requests over fast broadband.
• Mobile apps live for years in the wild; old versions must keep working. Web is redeployed instantly.

The Backend-for-Frontend pattern (coined at SoundCloud) gives each client experience its own thin backend layer, usually owned by the team that builds that client:

BFF granularity: "one per platform" (iOS, Android, web) is common; many companies use one mobile BFF for both iOS and Android because their needs are nearly identical. A GraphQL layer (Section 7) can replace multiple BFFs by letting each client ask for exactly the fields it wants.

5. North-south vs east-west traffic: gateway vs service mesh

Data-center diagrams are usually drawn with the internet at the top and servers below, so:

• North-south traffic enters or leaves the data center: client ⇄ backend. This goes through the API gateway.
• East-west traffic flows between services inside the data center: Feed service → Graph service, Like service → Kafka, etc. At Instagram scale, east-west traffic is many times larger than north-south — one feed request may fan out to dozens of internal calls.

East-west traffic needs many of the same features (load balancing, retries, timeouts, mTLS, metrics, tracing) but routing it all through a central gateway would add a hop and create a giant bottleneck. Instead, a service mesh (Istio, Linkerd, Consul Connect) puts a small proxy — a sidecar, usually Envoy — next to every service instance. Services talk to localhost; the sidecar does discovery, load balancing, mTLS encryption and identity, retries, circuit breaking and telemetry. A central control plane pushes configuration to all sidecars.

They are complementary, and often built on the same proxy (Envoy) — the gateway is sometimes called the mesh's "ingress gateway".

6. Keeping the gateway from becoming a SPOF or bottleneck

Every single API request passes through the gateway, so if it's down, Instagram is down; if it's slow, everything is slow. Design principles:

6.1 Stateless and horizontally scaled

• Stateless: a gateway instance keeps no per-user state in memory that another instance would need. Session data lives in the token (JWT) or in a shared store; rate-limit counters live in Redis (plus small local buckets). Any instance can serve any request, so you scale by adding instances behind the load balancer and can kill any instance at any time.
• Many instances across availability zones and regions: e.g. assume peak 1.5M API requests/sec and one gateway instance handles ~15k req/s with headroom → ~100 instances, then add N+2 per zone and spread them across 3 zones in each region. See Multi-region.
• Redundant layers in front: DNS returns multiple IPs; anycast or multiple L4 load balancers; health checks remove dead gateway instances within seconds.

6.2 Fail sensibly when dependencies fail

• Auth without a network call: validating JWT signatures locally with cached public keys means the gateway keeps working even if the auth service hiccups.
• Rate-limiter outage: if Redis is unreachable, decide in advance whether to fail open (allow requests, with local per-instance limits as a safety net) or fail closed (reject). Most consumer products fail open for general traffic and closed for sensitive endpoints like login.
• Config errors are the #1 risk: a bad routing rule pushed globally takes down everything at once. Roll out gateway config like code: validation, staged rollout region by region, instant rollback.
• Isolation: separate gateway pools (or bulkheads) for very different traffic — e.g. long-lived WebSocket connections for DMs vs short HTTP calls — so a surge in one can't starve the other.

6.3 Latency overhead budget

The gateway adds an extra network hop plus processing. Typical costs:

Target: total gateway overhead of ~1–5 ms at p99 on a request whose end-to-end budget is ~200 ms. Avoid putting slow work (synchronous calls to many services, heavy transformations, large body buffering) in the generic gateway path. Media uploads should bypass the gateway entirely by using pre-signed URLs straight to object storage — the gateway only hands out the URL.

7. GraphQL gateways

GraphQL (created at Facebook, open-sourced in 2015) lets the client send a query describing exactly the fields it wants; the server returns exactly that shape. A GraphQL gateway exposes one unified schema and resolves each field by calling the appropriate backend service.

query ProfileScreen {
  user(username: "natgeo") {
    fullName
    avatarUrl(size: 150)
    followerCount
    isFollowedByViewer
    posts(first: 12) { edges { node { id thumbnailUrl likeCount } } }
    storyHighlights { id title coverUrl }
  }
}

The gateway's resolvers map user → User service, followerCount/isFollowedByViewer → Graph service, posts → Post service, storyHighlights → Stories service, run them in parallel, and assemble one response. This gives aggregation and per-client shaping without writing a separate BFF endpoint for every screen.

• Federation: in large companies each team owns a subgraph (part of the schema), and a federated gateway (e.g. Apollo Router) composes them into one supergraph.
• The N+1 problem: resolving author for each of 12 posts naively makes 12 calls; a DataLoader batches them into one getUsers([ids]) call.
• Protecting the gateway: clients can write expensive queries, so enforce depth limits, cost analysis, and persisted queries (the app sends a query hash; the server only runs pre-registered queries). Persisted queries also shrink request size and make GET-caching possible.
• Rate limiting by cost rather than by request count, since one query may be cheap or enormous.

Comparison with REST and gRPC is in API Design.

8. How Instagram uses its API gateway

8.1 One request, step by step: liking a post

Let's follow POST /v1/posts/987/likes from the app:

1. DNS + connection. The app already has a warm HTTP/2 (or HTTP/3/QUIC) connection to i.instagram.com, resolved via DNS to the nearest edge. No new handshake needed.
2. Load balancer. The L4 load balancer hashes the connection to one healthy gateway instance in the region.
3. TLS termination. The gateway decrypts the request. It now sees method, path, headers and body.
4. Assign a request ID and start a trace. X-Request-Id: 7f3c… and a traceparent header are added; a timer starts.
5. Basic hygiene. Body size within limits? Content-Type allowed? Client version still supported (very old app builds get 426 Upgrade Required / a force-upgrade message)?
6. Authenticate. Parse Authorization: Bearer …, verify signature and expiry with the cached public key → user ID 1234, device ID d-55. Invalid → 401; the app uses its refresh token to get a new access token.
7. Rate limit. Check buckets in Redis: the general per-user request limit and the action-specific "likes per hour" limit used against spam bots. Over the limit → 429 with Retry-After.
8. Validate. Is 987 a well-formed post ID? Is an Idempotency-Key present for this write?
9. Route. The routing table says /v1/posts/{id}/likes → Like & Comment Service. Service discovery returns healthy instances; the gateway picks one (least-outstanding-requests) and forwards over a pooled connection, adding X-User-Id: 1234.
10. Service does the real work. The Like service checks object-level permission (is the post visible to 1234? is 1234 blocked by the author?), writes the like, increments the counter in Redis, and publishes a like.created event to Kafka (consumed by the Notification service to push "user liked your photo").
11. Response path. The gateway receives 200 {"liked": true, "like_count": 10431}, strips internal headers, adds rate-limit headers, compresses, and returns it.
12. Observe. The gateway asynchronously emits an access-log line and metrics (route=likes.create, status=200, latency=18 ms) and closes its trace span.

8.2 Rough numbers

Assume 500M daily active users, each making ~200 API calls/day (feed pages, stories, likes, prefetch, telemetry):

500M × 200 = 100 billion API requests/day
100e9 / 86,400 s ≈ 1.16M requests/s average; assume peak ≈ 2–3× → ~3M req/s
If one gateway instance comfortably handles ~20k req/s at p99 < 5 ms:
   3M / 20k = 150 instances at peak, ×1.5 headroom for zone failure ≈ 225 instances worldwide
Gateway overhead 2 ms × 100B requests/day ≈ 200M seconds of added latency per day —
   which is why every millisecond in the gateway path is engineered away.

Note what does not go through the gateway: photo/video bytes (served by the CDN and uploaded directly to object storage via pre-signed URLs). This keeps the gateway handling small JSON requests only.

9. Trade-offs & common pitfalls

Common pitfalls

• Trusting the gateway for object-level authorization — services must still check "can this user see this post?".
• Trusting identity headers from the outside. If a client sends X-User-Id: 1, the gateway must strip it and set its own; services must accept identity only from the gateway/mesh (mTLS).
• Stateful gateways (in-memory sessions) that break when an instance dies or traffic shifts.
• Routing big uploads through the gateway — ties up connections and memory; use pre-signed URLs.
• Missing timeouts to upstreams — one slow service exhausts the gateway's connection pool and takes down unrelated routes. Set per-route timeouts and use bulkheads.
• Retrying non-idempotent requests (POST) at the gateway can create duplicate posts or comments unless idempotency keys are used (see API Design).
• Logging sensitive data (tokens, passwords, full request bodies) in access logs.
• Global config pushes without staging — roll out gateway changes gradually.

10. Real-world technologies

11. Interview corner

12. Check your understanding

Related lessons

• Load Balancing — the layer in front of the gateway and the algorithms it uses for upstreams
• Rate Limiting — the algorithms behind the gateway's 429s
• Authentication & Authorization — tokens the gateway validates and the checks services must still do
• API Design — REST/gRPC/GraphQL contracts the gateway exposes and translates
• Microservices & Stateless Services — the services behind the gateway
• Resilience patterns — timeouts, retries and circuit breakers at the gateway and mesh
• Monitoring, Logging & Tracing — what to do with the gateway's logs, metrics and traces
• End-to-End Request Flows — see the gateway's place in every Instagram flow