Load Balancing

Load balancing distributes incoming traffic across multiple servers so no single machine is overwhelmed. It is the foundation of horizontal scaling — without it, you cannot add more servers to handle more traffic. Every production system uses some form of load balancing, from DNS round-robin to dedicated L7 load balancers like Nginx, HAProxy, or AWS ALB.

Why This Matters

Load balancing is used in virtually every production system. It is one of the first things you add when scaling beyond a single server. In interviews, discussing load balancing strategies shows you understand distributed system fundamentals.

The Building Blocks

• Round Robin: Requests are distributed to servers sequentially. Simple and fair if servers have equal capacity.
• Weighted Round Robin: Servers with more capacity get proportionally more requests. Useful for heterogeneous server fleets.
• Least Connections: New requests go to the server with the fewest active connections. Best for long-lived connections (WebSockets).
• IP Hash: The client's IP determines which server handles the request. Provides session affinity without cookies.
• Layer 4 vs Layer 7: L4 load balancers route by IP/port (faster, used for TCP traffic). L7 load balancers route by HTTP content (URL, headers, cookies — smarter but slower).
• Health checks: Load balancers periodically ping backend servers. Unhealthy servers are removed from the pool until they recover.

Under the Hood

A load balancer sits between clients and backend servers. When a request arrives, the LB selects a healthy server using its configured algorithm and forwards the request. The server processes the request and sends the response back through the LB (or directly to the client in DSR mode).

For high availability, deploy load balancers in pairs (active-passive or active-active) to avoid the LB itself becoming a single point of failure.

How Companies Actually Do This

AWS ALB: Layer 7 load balancer that routes based on URL path (/api → API servers, /static → static servers), supports WebSockets, and integrates with auto-scaling.

Nginx: The world's most popular reverse proxy/load balancer. Handles millions of concurrent connections with minimal memory.

Google Cloud: Uses Maglev, a custom Layer 4 load balancer that distributes traffic across global backend pools using consistent hashing.

Common Pitfalls

1. Not configuring health checks — traffic continues going to dead servers
2. Using round robin with heterogeneous servers — overloads weaker machines
3. Making the load balancer a single point of failure
4. Not considering connection draining during deployments

Interview Questions Worth Practicing

1. What load balancing algorithms do you know? When would you use each?
2. What is the difference between Layer 4 and Layer 7 load balancing?
3. How do you avoid the load balancer becoming a single point of failure?
4. What is sticky sessions and when is it needed?
5. How does health checking work?

The Tradeoffs

• L4 vs L7: L4 is faster (no HTTP parsing) but L7 is smarter (content-based routing).
• Sticky sessions vs stateless: Sticky sessions simplify session management but reduce load distribution effectiveness.
• Hardware vs software LB: Hardware LBs (F5) handle more traffic; software LBs (Nginx, HAProxy) are cheaper and more flexible.

How to Explain This in an Interview

Here is how I would explain Load Balancing in a system design interview:

Load balancing is how you scale horizontally. Put a load balancer in front of N identical servers. When a request arrives, the LB picks a healthy server — round-robin for simplicity, least-connections when request durations vary, consistent hashing when you need session affinity. Three things interviewers want to hear: first, health checks — the LB pings backends and removes dead ones from the pool. Second, redundancy — run LBs in active-passive pairs so the LB itself is not a SPOF. Third, the L4 vs L7 decision — use L4 for raw TCP and database connections, L7 when you need to route based on URL paths or HTTP headers. In .NET systems, you would typically sit behind an Azure Application Gateway or Nginx, with Kestrel handling the application layer.

Related Topics

• Proxy vs Reverse Proxy
• Consistent Hashing
• Scalability
• Design a Load Balancer
• Load Balancing Algorithms

Load Balancing .NET Applications

When deploying ASP.NET Core behind a load balancer, there are specific configurations you need:

Forwarded Headers: Behind a reverse proxy (Nginx, Azure Application Gateway), your app does not see the client's real IP. Configure ForwardedHeaders middleware:

builder.Services.Configure<ForwardedHeadersOptions>(options =>
    options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
);
app.UseForwardedHeaders();

Health Checks: ASP.NET Core has built-in health check endpoints that load balancers use to detect unhealthy instances:

builder.Services.AddHealthChecks()
    .AddSqlServer(connectionString)    // Check database
    .AddRedis(redisConnection)         // Check cache
    .AddUrlGroup(new Uri("https://api.stripe.com"), "stripe"); // Check dependencies

app.MapHealthChecks("/healthz");

Azure-specific: Azure App Service and Azure Kubernetes Service (AKS) provide built-in load balancing. App Service auto-scales based on CPU, memory, or HTTP queue length. For .NET developers, this means zero load balancer configuration — Azure handles it.

Kestrel tuning: The Kestrel web server in ASP.NET Core handles 1M+ concurrent connections. For high-throughput scenarios, tune MaxConcurrentConnections and MaxRequestBodySize in your configuration.

Original DosiBridge notes for Load Balancing

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Load Balancing is to connect the concept to a traffic-management problem where overload behavior matters more than the happy-path diagram. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Packet Path: For Load Balancing, decide the packet path before choosing tools; otherwise the design looks clean but fails when load changes.
• Timeout Budget: For Load Balancing, decide the timeout budget before choosing tools; otherwise the design looks clean but fails when balancing changes.
• TLS Boundary: For Load Balancing, decide the TLS boundary before choosing tools; otherwise the design looks clean but fails when load changes.
• Proxy Behavior: For Load Balancing, decide the proxy behavior before choosing tools; otherwise the design looks clean but fails when balancing changes.
• Backpressure: For Load Balancing, decide the backpressure before choosing tools; otherwise the design looks clean but fails when load changes.

Production review question

If a team adopts Load Balancing, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track load-balancing health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Load Balancing, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.