Design a Distributed Cache

Requirements

Functional: get(key), set(key, value, ttl), eviction when full. Non-functional: sub-millisecond reads; must scale beyond what fits in one machine's memory; cache misses should degrade gracefully, not cascade into a database outage.

Where the cache sits

Eviction policy

LRU (Least Recently Used) is the default choice for general-purpose caching — it approximates "keep what's likely to be accessed again" cheaply. LFU (Least Frequently Used) can do better for access patterns with a stable "hot set," but costs more to track. Most systems (Redis included) default to LRU-family policies precisely because the implementation cost/benefit favors it for the common case.

Cache-aside vs. write-through

• Cache-aside (lazy loading): the app checks the cache, and on a miss, reads from the DB and populates the cache itself (as shown above). Simple, and only caches what's actually requested — but the first request for any key is always a miss.
• Write-through: every write goes to the cache and the DB together, so the cache is never stale for written keys. Adds write latency and can waste cache space on data that's never read again.

Most systems default to cache-aside for reads and handle writes by simply invalidating (deleting) the cached key rather than trying to keep it in sync — deleting is simpler and safer than updating, since a partially-failed update can leave stale data silently in the cache.

Follow-up: what happens when a cache node fails?

With consistent hashing, only the keys that node owned are affected — they become misses until the ring rebalances onto surviving nodes, and traffic falls through to the database for just that slice of keys rather than the whole cache going cold at once. This is the same resilience property discussed in the Distributed Key-Value Store problem.