System Design Fundamentals

What System Design Actually Is

System design is the process of defining the architecture, components, data flow, and interfaces of a system to satisfy a given set of requirements. It bridges the gap between a product idea and a working production system that serves real users at scale.

When someone says "design a URL shortener" or "design a chat application," they are not asking you to write code. They are asking you to reason about how the pieces fit together: which databases store what data, how services communicate, where caches sit, how the system handles failure, and what happens when traffic grows by 100x.

This is not an academic exercise. Every outage at a major company traces back to a system design decision. When Amazon's S3 went down in 2017, it was because a single human error cascaded through a system that lacked proper blast radius controls. When Facebook went offline for six hours in 2021, it was a BGP misconfiguration combined with insufficient out-of-band access to recover. These failures are design failures.

Why Every Engineer Needs This

Junior engineers write functions. Senior engineers design systems. The difference between a mid-level developer and a staff engineer is often the ability to reason about system-level concerns: What happens when the database goes down? How do we handle 10x traffic on Black Friday? Where do we put the cache, and what consistency guarantees do we need?

System design knowledge is also the primary signal in senior engineering interviews at companies like Google, Meta, Amazon, and Stripe. The interview is not about memorizing architectures. It is about demonstrating structured thinking under ambiguity.

The Building Blocks

Every system, from a simple CRUD app to Netflix's streaming platform, is composed of a small set of recurring building blocks. Mastering these building blocks gives you a vocabulary for reasoning about any architecture.

Compute

Application servers run your business logic. They receive requests, process them, and return responses. The key design decisions around compute are:

• Stateless vs. stateful: Stateless servers store no session data locally, which means any server can handle any request. This makes horizontal scaling straightforward. Stateful servers (like WebSocket servers maintaining persistent connections) require sticky sessions or connection-aware routing.
• Synchronous vs. asynchronous: Synchronous processing blocks until the work is done. Asynchronous processing enqueues the work and returns immediately, processing it later. Order confirmation emails do not need to block the checkout flow.
• Scaling strategy: Horizontal scaling (more machines) is preferred over vertical scaling (bigger machines) for most workloads, because it avoids single points of failure.

Storage

Databases persist your data. The fundamental choice is between relational databases (PostgreSQL, MySQL) and non-relational databases (MongoDB, Cassandra, DynamoDB). This is not a religious debate. The right answer depends on your access patterns, consistency requirements, and scale.

Relational databases excel when you need ACID transactions, complex queries with joins, and strong consistency. Non-relational databases excel when you need flexible schemas, horizontal write scaling, or specific data models (document, wide-column, graph, key-value).

Beyond the primary database, most production systems use multiple storage layers: a cache (Redis, Memcached) for hot data, an object store (S3) for blobs, a search index (Elasticsearch) for full-text queries, and a data warehouse (BigQuery, Snowflake) for analytics.

Networking

Every distributed system is a set of machines communicating over a network. The networking layer includes:

• Load balancers distribute incoming traffic across multiple servers. Layer 4 (TCP) load balancers route based on IP and port. Layer 7 (HTTP) load balancers can route based on URL path, headers, or request content.
• DNS translates domain names to IP addresses. It is the first thing that happens when a user types a URL. DNS-based load balancing (round-robin DNS, GeoDNS) is a coarse-grained distribution mechanism.
• CDNs cache static content at edge locations close to users, reducing latency for assets like images, CSS, and JavaScript. Cloudflare, Akamai, and CloudFront are examples.
• API gateways sit in front of your services and handle cross-cutting concerns: authentication, rate limiting, request routing, and protocol translation.

Caching

Caching stores frequently accessed data in a faster storage layer (typically memory) to reduce latency and database load. A well-placed cache can reduce response times from 100ms to under 1ms.

The key caching decisions are: what to cache (hot data, computed results, session data), where to cache (client-side, CDN, application-level, database-level), and how to invalidate (TTL-based, event-driven, write-through, write-behind).

Cache invalidation is famously one of the two hard problems in computer science. Stale caches cause subtle bugs. A user updates their profile picture but still sees the old one. A product price changes but the cached page shows the old price. Getting invalidation right requires careful design.

Message Queues and Asynchronous Processing

Not every operation needs to happen in real time. Message queues (Kafka, RabbitMQ, SQS) decouple producers from consumers, enabling asynchronous processing. When a user uploads a video to YouTube, the upload returns immediately. Transcoding, thumbnail generation, and content moderation happen asynchronously via a processing pipeline.

Message queues also provide durability (the message persists even if the consumer crashes), load leveling (consumers process at their own pace), and fan-out (one message triggers multiple downstream actions).

The Design Process

A structured approach to system design prevents you from jumping to solutions before understanding the problem. Here is a framework that works in both interviews and production design reviews.

Step 1 — Clarify requirements. Ask what the system needs to do (functional requirements) and how well it needs to do it (non-functional requirements: latency, throughput, availability, consistency). "Design Twitter" is hopelessly vague. "Design a system that supports 500M users posting 500M tweets per day with a timeline that loads in under 200ms" gives you constraints to work with.

Step 2 — Estimate scale. Back-of-the-envelope calculations ground your design in reality. If you have 500M daily active users and each generates 10 read requests, that is 5 billion reads per day, roughly 58,000 reads per second. This tells you immediately that a single database will not suffice.

Step 3 — Define the high-level design. Draw the major components and their interactions: clients, load balancers, application servers, databases, caches, message queues. This is the skeleton of your system.

Step 4 — Deep dive into key components. Pick the 2-3 most interesting or challenging components and design them in detail. For a chat application, that might be the message delivery system and the presence indicator. For a URL shortener, it might be the ID generation scheme and the redirect path.

Step 5 — Address bottlenecks and failure modes. What happens when the primary database goes down? What if traffic spikes 10x? Where are the single points of failure? This is where you demonstrate senior-level thinking.

Common Patterns in Production

Certain patterns appear repeatedly across real-world systems:

• Read replicas: Separate read traffic from write traffic by replicating data to read-only database instances. Instagram uses this extensively with PostgreSQL.
• Sharding: Partition data across multiple database instances by a shard key (user ID, geographic region). Uber shards rides by city.
• CQRS (Command Query Responsibility Segregation): Use different models for reads and writes. The write model is optimized for consistency; the read model is optimized for query performance.
• Event sourcing: Store every state change as an immutable event rather than overwriting current state. This provides a complete audit trail and enables temporal queries. Banks and financial systems use this pattern extensively.
• Circuit breakers: Prevent cascading failures by detecting when a downstream service is unhealthy and failing fast instead of waiting for timeouts. Netflix's Hystrix library popularized this pattern.

Where to Start

If you are new to system design, start with the foundational concepts: scalability, availability, and the CAP theorem. Then move to specific building blocks: databases, caching, and load balancing. Finally, practice by designing real systems end-to-end. Each concept in this series builds on the previous one, and the problems section provides structured practice with real interview questions.

The goal is not to memorize architectures. It is to build intuition for the tradeoffs involved in every design decision. There is no single correct answer to any system design question. There are only tradeoffs, and the best engineers are the ones who can articulate them clearly.

Real-World Production Example

When Slack was building their messaging platform, they applied system design fundamentals in a sequence that many successful startups follow. They started with a simple architecture: a monolithic PHP application backed by MySQL, with Memcached for caching. This handled their first million users because the fundamentals were right — they separated compute from storage, cached aggressively, and used a load balancer to distribute traffic across application servers.

As Slack grew to tens of millions of daily active users, each fundamental became a scaling lever. They added read replicas to handle the read-heavy workload of loading channel histories. They introduced Solr for search because relational queries could not handle full-text search across billions of messages efficiently. They added Kafka for asynchronous processing of events like notifications, analytics, and search indexing. Each addition was a response to a specific bottleneck, not a premature optimization.

The most important decision Slack made was keeping their core message delivery path simple while adding complexity only where measured bottlenecks demanded it. Their message delivery still goes through a relatively straightforward path: WebSocket connection to a gateway server, gateway publishes to a message bus, and recipient gateway servers push to connected clients. The complexity lives in the supporting systems (search, notifications, compliance, analytics) that consume messages asynchronously. This separation of the fast path from the complex path is a fundamental design principle that every system architect should internalize.

Original DosiBridge notes for System Design Fundamentals

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study System Design Fundamentals is to connect the concept to a production service where growth exposes the first weak assumption in the original design. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• SLO: For System Design Fundamentals, decide the SLO before choosing tools; otherwise the design looks clean but fails when system changes.
• Capacity Headroom: For System Design Fundamentals, decide the capacity headroom before choosing tools; otherwise the design looks clean but fails when design changes.
• Failure Mode: For System Design Fundamentals, decide the failure mode before choosing tools; otherwise the design looks clean but fails when fundamentals changes.
• Degradation Path: For System Design Fundamentals, decide the degradation path before choosing tools; otherwise the design looks clean but fails when system changes.
• Operator Signal: For System Design Fundamentals, decide the operator signal before choosing tools; otherwise the design looks clean but fails when design changes.

Production review question

If a team adopts System Design Fundamentals, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track system-fundamentals health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For System Design Fundamentals, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.