Design a Distributed Cache: Redis Cluster, Sharding and Replication

Problem Statement

Design a distributed in-memory cache (similar to Redis or Memcached) that provides sub-millisecond read/write latency across a cluster of nodes. The system must handle node failures gracefully, distribute data evenly, and support configurable eviction policies and TTLs.

Requirements

Functional

• GET/SET/DELETE operations with sub-millisecond latency
• Support TTL (time-to-live) per key for automatic expiration
• Configurable eviction policies: LRU, LFU, or random
• Cache invalidation via explicit delete or pub/sub broadcast

Non-Functional

• Latency: <1ms for cache hits, <5ms for cache misses (passthrough to DB)
• Throughput: 500K ops/second per node
• Availability: Cluster survives individual node failures without data loss for replicated keys
• Scale: Horizontally scalable to 100+ nodes, petabytes of cached data

Core Architecture

1. Cache Node -- Each node runs a single-threaded event loop (like Redis) for lock-free execution. Stores data in a hash table with a doubly-linked list for LRU ordering. Supports multiple data types: strings, lists, sets, sorted sets, and hashes.

2. Consistent Hash Ring -- Maps keys to nodes using a hash ring with 150+ virtual nodes per physical node to ensure even distribution. When a node joins or leaves, only 1/N of keys are redistributed. Client libraries compute the target node locally, avoiding a central router.

3. Replication Manager -- Each primary node asynchronously replicates writes to 1-2 replica nodes. On primary failure, a replica is promoted. Uses a replication log with sequence numbers so replicas can catch up after brief network partitions.

4. Cluster Coordinator -- A lightweight gossip protocol (like Redis Cluster) where nodes exchange heartbeats every second. Detects node failures within 5-10 seconds. Maintains and propagates the hash ring topology. No single point of failure.

Database Choice

The cache itself is purely in-memory with optional append-only file (AOF) persistence for durability across restarts. The AOF logs every write command and is fsync'd every second (configurable). For cluster metadata, each node maintains its own copy of the hash ring propagated via gossip protocol -- no external database needed. Monitoring metrics are exported to Prometheus.

Key API Endpoints

GET /cache/\{key\}
  -> Returns: \{ value: "...", ttl_remaining_ms: 4500 \}

PUT /cache/\{key\}
  -> Body: \{ value: "...", ttl_ms: 60000 \}

DELETE /cache/\{key\}
  -> Returns: \{ deleted: true \}

Scaling Insight

The most important scaling decision is client-side routing via consistent hashing. Instead of routing all requests through a proxy (which becomes a bottleneck), each client library maintains a local copy of the hash ring and routes directly to the correct cache node. This eliminates the proxy hop, halves latency, and doubles throughput. The hash ring is updated via cluster topology change events pushed to clients.

Key Tradeoffs

Original DosiBridge notes for Design a Distributed Cache: Redis Cluster, Sharding and Replication

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Design a Distributed Cache: Redis Cluster, Sharding and Replication is to connect the concept to a read-heavy path where the cache can either protect the database or serve confusing stale data. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Core Entity: For Design a Distributed Cache: Redis Cluster, Sharding and Replication, decide the core entity before choosing tools; otherwise the design looks clean but fails when design changes.
• Read/Write Path: For Design a Distributed Cache: Redis Cluster, Sharding and Replication, decide the read/write path before choosing tools; otherwise the design looks clean but fails when distributed changes.
• Contention Point: For Design a Distributed Cache: Redis Cluster, Sharding and Replication, decide the contention point before choosing tools; otherwise the design looks clean but fails when cache changes.
• Abuse Path: For Design a Distributed Cache: Redis Cluster, Sharding and Replication, decide the abuse path before choosing tools; otherwise the design looks clean but fails when design changes.
• Recovery Workflow: For Design a Distributed Cache: Redis Cluster, Sharding and Replication, decide the recovery workflow before choosing tools; otherwise the design looks clean but fails when distributed changes.

Design a Distributed Cache: Redis Cluster, Sharding and Replication correctness boundary

For this problem, separate the user-visible action from background work. The correctness boundary is the smallest state transition that must not be duplicated, lost, or reordered. Naming that boundary makes the solution more original than a stock API/database/cache template.

Design a Distributed Cache: Redis Cluster, Sharding and Replication scaling bottleneck

The first bottleneck should be described in the language of this system: a hot timeline, a contested seat, a busy wallet ledger, a media-processing queue, or a search index update delay. That bottleneck decides which component deserves the deepest interview discussion.

Production review question

If Design a Distributed Cache: Redis Cluster, Sharding and Replication suddenly receives 10x traffic, which entity becomes contested first, and what queue, lock, shard, or cache protects that entity without hiding correctness bugs?

Practical metric to track

Track design-cache health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Design a Distributed Cache: Redis Cluster, Sharding and Replication, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.

Sources

• Design a Distributed Cache -- Reference
• Source: System-Design-Overview