Design a Distributed Cache

Intermediate Storage & databases

The problem

Design a distributed in-memory cache (like a Redis or Memcached cluster) that many application servers share to avoid slow database reads.

You are in a 45-minute interview. The interviewer wants more than 'use Redis'.

Your applications already have a relational database that is the source of truth.

The cache must keep working as machines are added, removed or crash.

Drive the conversation. Ask questions before drawing anything. State your assumptions out loud so they can be corrected early.
Use numbers. Turn the givens into requests per second, storage and bandwidth. Every later choice should point back to one of them.
Start simple, then evolve. Get a working version on the board first. Add caches, sharding and queues only when a number or a requirement asks for them.
Go deep on the hard parts. For this problem the hard parts are choosing the placement scheme, keeping the cache and database consistent, and surviving node loss without flooding the database.
Name the trade-offs. Say what you chose, what you gave up, and what would make you change your mind.
Think about failure. Point out single points of failure and say what happens when a node, a cache or a whole zone goes down.

LRU

Least Recently Used: evict the entry that has not been touched for the longest time.

TTL

Time to live: after this long the entry expires.

Cache-aside

The application checks the cache, and on a miss reads the database and fills the cache.

Cache stampede

Many requests miss on the same key and all hit the database at once.

Hot key

One key getting so much traffic that a single node cannot cope.

200M x 20% x 1 KB = about 40 GB hot data
Memory access is roughly 100 ns; a network hop to a cache is about 0.5 ms; a database query is 5 to 50 ms
Add 30-50% overhead for metadata

Decisions you confirm will collect here as you go.

Your design so far