Case Study: Design an API Gateway

Mental Model

Connecting isolated components into a resilient, scalable, and observable distributed web.

An API Gateway is the single entry point for all clients. It handles cross-cutting concerns like Authentication, Rate Limiting, and Request Routing.

1. Requirement Clarification

graph TD
    App[Application Server] -->|Read Request| Cache[(Redis Cache)]
    Cache -- Cache Miss --> DB[(Primary Database)]
    DB -- Return Data --> App
    App -- Write Data --> Cache

Functional

• Route requests to the correct microservice.
• Authenticate and Authorize requests.
• Aggregate responses (Fan-out/Fan-in).

Non-Functional

• High Availability: If the Gateway is down, the whole system is down.
• Ultra-low Latency: The gateway adds a "hop." It must be as fast as possible (< 10ms).
• Security: Protect against DDoS and SQL Injection.

2. High-Level Architecture

1. Client $\rightarrow$ LB $\rightarrow$ API Gateway.
2. API Gateway $\rightarrow$ Service Discovery (to find service IPs).
3. API Gateway $\rightarrow$ Auth Service.
4. API Gateway $\rightarrow$ Microservices.

3. Scaling the Gateway

The gateway should be Stateless. Use a pool of instances behind a Layer 4 Load Balancer. Use Configuration Management (like Etcd or Zookeeper) to update routing rules without restarting the gateway.

4. Performance: Synchronous vs. Asynchronous

• Blocking (I/O): One thread per request. Easy but scales poorly.
• Non-blocking (Event-driven): Uses event loops (e.g., Netty, Nginx). Handles thousands of connections per thread. Preferred for scale.

Final Takeaway

The API Gateway is a Centralized Control Plane. It allows you to enforce global policies without changing code in individual microservices.

Technical Trade-offs: Database Choice

Key Takeaways

• Route requests to the correct microservice.
• Authenticate and Authorize requests.
• Aggregate responses (Fan-out/Fan-in).

Production Readiness Checklist

Before deploying this architecture to a production environment, ensure the following Staff-level criteria are met:

• High Availability: Have we eliminated single points of failure across all layers?
• Observability: Are we exporting structured JSON logs, custom Prometheus metrics, and OpenTelemetry traces?
• Circuit Breaking: Do all synchronous service-to-service calls have timeouts and fallbacks (e.g., via Resilience4j)?
• Idempotency: Can our APIs handle retries safely without causing duplicate side effects?
• Backpressure: Does the system gracefully degrade or return HTTP 429 when resources are saturated?

Read Next

• Case Study: Design a Fraud Detection System
• Deep Dive: Rate Limiting Algorithms