Design a Rate Limiter

Requirements

Functional: limit each client (by API key, user ID, or IP) to N requests per time window; reject requests over the limit with a clear signal (HTTP 429). Non-functional: the limiter itself must not become the bottleneck — checking a limit should add negligible latency; must work correctly across multiple API server instances (not just one process's memory).

Algorithm choice

Architecture

Why Redis, and why a Lua script specifically

Rate limiting needs a shared counter across every API server instance — in-memory counters per-process would let a client get N requests per server instead of N total. Redis gives shared, low-latency state. The check-and-decrement must be atomic: if you read the count, then decrement it in two separate round-trips, two concurrent requests can both read "1 token left" and both proceed, letting the client burst past their limit (a classic race condition). A Lua script runs both steps as one atomic operation on the Redis server.

Placement: gateway vs. application

Rate limiting at the API gateway protects every backend service uniformly and rejects abusive traffic before it costs you any backend capacity. Rate limiting inside a specific service is useful for business-rule limits (e.g. "3 free trials per account") that don't apply globally. Most systems use both, at different layers.

Follow-up: what happens when Redis is down?

This is the fail-open vs. fail-closed tradeoff: fail-open (let all requests through if the limiter is unreachable) prioritizes availability over strict limits; fail-closed (reject everything) prioritizes protecting backend capacity over availability. Most production systems fail-open for rate limiting specifically, since an outage in the limiter shouldn't take down the whole API.