Distributed Caching
A single Redis server can only hold as much data as its RAM allows. Distributed caching (Redis Cluster, Memcached with consistent hashing) scales.
Quick answer: Distributed Caching
Distributed Caching is covered here as a practical system design concept. A single Redis server can only hold as much data as its RAM allows. Distributed caching (Redis Cluster, Memcached with consistent hashing) scales. Use it to understand the Caching idea, where it appears in production systems, and how to discuss it clearly in interviews.
Focus on the problem being solved, the main design choice, the failure mode, and the tradeoff you would defend in front of a senior engineer. This page is part of the DosiBridge system design knowledge graph, so it links related concepts, interview problems, tradeoffs, and references instead of leaving the topic isolated.
When You Need Distributed Caching
A single Redis server can only hold as much data as its RAM allows. Distributed caching (Redis Cluster, Memcached with consistent hashing) scales horizontally to terabytes of cached data across dozens of nodes.
What It Is
Distributed caching spreads cached data across multiple server nodes, providing higher capacity, availability, and throughput than a single cache server. Data is partitioned using consistent hashing so each node holds a portion of the cache.
How It Works
In a Redis Cluster with 6 nodes, the key space (16384 hash slots) is divided equally. When you SET user:123, the key is hashed to a slot (e.g., slot 5234), which maps to node 3. GET user:123 routes to the same node. If node 3 fails, its replica takes over.
The beauty: adding a new node only moves a fraction of keys (consistent hashing). The cluster rebalances automatically.
The Decision Framework
- Partitioning: Data is distributed across nodes using consistent hashing. Each key maps to a specific node.
- Replication: Critical cached data can be replicated to multiple nodes for availability. If one node fails, replicas serve the data.
- Client-side sharding: The client library determines which node to query (Memcached approach).
- Server-side sharding: The cache cluster handles routing internally (Redis Cluster approach).
- Cache stampede protection: When a popular key expires, thousands of requests simultaneously miss. Use locking (only one request fetches from DB) or probabilistic early recomputation.
What the Industry Uses
Facebook runs the world's largest Memcached deployment — thousands of servers caching social graph data.
Twitter uses a massive Redis cluster to cache timelines, user sessions, and rate limiting counters.
Slack uses Redis Cluster for presence information (who is online) across millions of users.
Performance and Tradeoffs
- Complexity: Distributed caching requires managing multiple nodes, replication, and failover.
- Network hops: Each cache request requires a network round trip (unlike local in-process caching).
- Consistency: In a distributed cache, stale reads are possible during node failures or rebalancing.
Mistakes Engineers Make
- Not handling cache node failures — requests to the failed node time out
- Not using consistent hashing — adding a node invalidates the entire cache
- Caching too aggressively — memory is expensive at scale
Practice These Interview Questions
- How does distributed caching differ from a single cache?
- How do you partition data in a distributed cache?
- What happens when a cache node fails?
- What is a cache stampede and how do you prevent it?
Original DosiBridge notes for Distributed Caching
This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Distributed Caching is to connect the concept to a production service where growth exposes the first weak assumption in the original design. That framing changes the design discussion from definitions to operating choices.
What makes this topic distinct
- Freshness Window: For Distributed Caching, decide the freshness window before choosing tools; otherwise the design looks clean but fails when distributed changes.
- Eviction Pressure: For Distributed Caching, decide the eviction pressure before choosing tools; otherwise the design looks clean but fails when caching changes.
- Hot Key Behavior: For Distributed Caching, decide the hot-key behavior before choosing tools; otherwise the design looks clean but fails when distributed changes.
- Cache Stampede: For Distributed Caching, decide the cache stampede before choosing tools; otherwise the design looks clean but fails when caching changes.
- Invalidation Owner: For Distributed Caching, decide the invalidation owner before choosing tools; otherwise the design looks clean but fails when distributed changes.
Production review question
If a team adopts Distributed Caching, what observable signal proves it helped instead of only making the architecture diagram more fashionable?
Practical metric to track
Track distributed-caching health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Distributed Caching, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.