Load Balancing Algorithms

Source-code implementations of 5 load balancing algorithms: Round Robin, Weighted Round Robin, Least Connections, Least Response Time, and IP Hash.

Quick answer: Load Balancing Algorithms

Load Balancing Algorithms is covered here as a practical system design resource. Source-code implementations of 5 load balancing algorithms: Round Robin, Weighted Round Robin, Least Connections, Least Response Time, and IP Hash. Use it as a curated reference while connecting the original engineering lesson to practical system design decisions.

Focus on the problem being solved, the main design choice, the failure mode, and the tradeoff you would defend in front of a senior engineer. This page is part of the DosiBridge system design knowledge graph, so it links related concepts, interview problems, tradeoffs, and references instead of leaving the topic isolated.

Load Balancing Algorithms — Code Lab

Load balancers distribute incoming traffic across multiple servers. The algorithm used determines how requests are assigned. Here are 5 common algorithms with working source-code implementations.

Algorithm Comparison

Algorithm	Time	When to Use	Session Affinity
Round Robin	O(1)	Equal-capacity servers	No
Weighted Round Robin	O(1)	Mixed-capacity servers	No
Least Connections	O(N)	Long-lived connections	No
Least Response Time	O(N)	Latency-sensitive apps	No
IP Hash	O(1)	Stateful sessions	Yes

Round Robin

Distributes requests sequentially across servers. Server 1, then Server 2, then Server 3, then back to Server 1. Simple and fair when all servers have equal capacity.

System architecture diagram for Load Balancing Algorithms showing how services, databases, and caches connect — System architecture for Load Balancing Algorithms

Time Complexity: O(1)
Space Complexity: O(N)
Best For: Homogeneous server fleet with equal capacity

Java

java

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class RoundRobin {
    private List<String> servers;
    private AtomicInteger index;

    public RoundRobin(List<String> servers) {
        this.servers = servers;
        this.index = new AtomicInteger(-1);
    }

    public String getNextServer() {
        int currentIndex = index.incrementAndGet() % servers.size();
        return servers.get(currentIndex);
    }

    public static void main(String[] args) {
        List<String> servers = List.of("Server1", "Server2", "Server3");
        RoundRobin roundRobinLB = new RoundRobin(servers);

        for (int i = 0; i < 6; i++) {
            System.out.println(roundRobinLB.getNextServer());
        }
    }
}

Python

python

Weighted Round Robin

Step-by-step diagram showing how Load Balancing Algorithms processes a request from start to finish — How Load Balancing Algorithms works step by step

Like round robin, but servers with higher capacity receive proportionally more requests. A server with weight 3 gets 3 requests for every 1 request to a server with weight 1.

Time Complexity: O(1)
Space Complexity: O(N)
Best For: Heterogeneous servers with different capacities

Java

java

import java.util.List;

public class WeightedRoundRobin {
    private List<String> servers;
    private List<Integer> weights;
    private int currentIndex;
    private int currentWeight;

    public WeightedRoundRobin(List<String> servers, List<Integer> weights) {
        this.servers = servers;
        this.weights = weights;
        this.currentIndex = -1;
        this.currentWeight = 0;
    }

    public String getNextServer() {
        while (true) {
            currentIndex = (currentIndex + 1) % servers.size();
            if (currentIndex == 0) {
                currentWeight--;
                if (currentWeight <= 0) {
                    currentWeight = getMaxWeight();
                }
            }
            if (weights.get(currentIndex) >= currentWeight) {
                return servers.get(currentIndex);
            }
        }
    }

    private int getMaxWeight() {
        return weights.stream().max(Integer::compare).orElse(0);
    }

    public static void main(String[] args) {
        List<String> servers = List.of("Server1", "Server2", "Server3");
        List<Integer> weights = List.of(5, 1, 1);
        WeightedRoundRobin weightedRoundRobinLB = new WeightedRoundRobin(servers, weights);

        for (int i = 0; i < 7; i++) {
            System.out.println(weightedRoundRobinLB.getNextServer());
        }
    }
}

Python

python

class WeightedRoundRobin:
    def __init__(self, servers, weights):
        self.servers = servers
        self.weights = weights
        self.current_index = -1
        self.current_weight = 0

    def get_next_server(self):
        while True:
            self.current_index = (self.current_index + 1) % len(self.servers)
            if self.current_index == 0:
                self.current_weight -= 1
                if self.current_weight <= 0:
                    self.current_weight = max(self.weights)
            if self.weights[self.current_index] >= self.current_weight:
                return self.servers[self.current_index]

# Example usage
servers = ["Server1", "Server2", "Server3"]
weights = [5, 1, 1]
load_balancer = WeightedRoundRobin(servers, weights)

for i in range(7):
    server = load_balancer.get_next_server()
    print(f"Request {i + 1} -> {server}")

Comparison table for Load Balancing Algorithms contrasting approaches, tradeoffs, and when to use each — Comparing key aspects of Load Balancing Algorithms

Least Connections

Routes each new request to the server with the fewest active connections. Adapts dynamically to server load — slow servers naturally get fewer requests.

Time Complexity: O(N) or O(log N) with heap
Space Complexity: O(N)
Best For: Long-lived connections (WebSockets, database connections)

Java

java

import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class LeastConnections {
    private Map<String, Integer> serverConnections;

    public LeastConnections(List<String> servers) {
        serverConnections = new HashMap<>();
        for (String server : servers) {
            serverConnections.put(server, 0);
        }
    }

    public String getNextServer() {
        return serverConnections.entrySet().stream()
                .min(Map.Entry.comparingByValue())
                .map(Map.Entry::getKey)
                .orElse(null);
    }

    public void releaseConnection(String server) {
        serverConnections.computeIfPresent(server, (k, v) -> v > 0 ? v - 1 : 0);
    }

    public static void main(String[] args) {
        List<String> servers = List.of("Server1", "Server2", "Server3");
        LeastConnections leastConnectionsLB = new LeastConnections(servers);

        for (int i = 0; i < 6; i++) {
            String server = leastConnectionsLB.getNextServer();
            System.out.println(server);
            leastConnectionsLB.releaseConnection(server);
        }
    }
}

Python

python

import random

class LeastConnections:
    def __init__(self, servers):
        self.servers = {server: 0 for server in servers}

    def get_next_server(self):
        # Find the minimum number of connections
        min_connections = min(self.servers.values())
        # Get all servers with the minimum number of connections
        least_loaded_servers = [server for server, connections in self.servers.items() if connections == min_connections]
        # Select a random server from the least loaded servers
        selected_server = random.choice(least_loaded_servers)
        self.servers[selected_server] += 1
        return selected_server

    def release_connection(self, server):
        if self.servers[server] > 0:
            self.servers[server] -= 1

# Example usage
servers = ["Server1", "Server2", "Server3"]
load_balancer = LeastConnections(servers)

for i in range(6):
    server = load_balancer.get_next_server()
    print(f"Request {i + 1} -> {server}")
    load_balancer.release_connection(server)

Data flow diagram for Load Balancing Algorithms showing how requests and responses move through the system — Data flow through Load Balancing Algorithms

Least Response Time

Routes to the server with the lowest average response time AND fewest active connections. Combines load awareness with performance awareness.

Time Complexity: O(N)
Space Complexity: O(N)
Best For: Latency-sensitive applications where server performance varies

Java

java

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class LeastResponseTime {
    private List<String> servers;
    private List<Double> responseTimes;

    public LeastResponseTime(List<String> servers) {
        this.servers = servers;
        this.responseTimes = new ArrayList<>(servers.size());
        for (int i = 0; i < servers.size(); i++)
            responseTimes.add(0.0);
    }

    public String getNextServer() {
        double minResponseTime = responseTimes.get(0);
        int minIndex = 0;
        for (int i = 1; i < responseTimes.size(); i++) {
            if (responseTimes.get(i) < minResponseTime) {
                minResponseTime = responseTimes.get(i);
                minIndex = i;
            }
        }
        return servers.get(minIndex);
    }

    public void updateResponseTime(String server, double responseTime) {
        int index = servers.indexOf(server);
        responseTimes.set(index, responseTime);
    }

    public static double simulateResponseTime(String server) {
        // Simulating response time with random delay
        Random random = new Random();
        double delay = 0.1 + (1.0 - 0.1) * random.nextDouble();
        try {
            Thread.sleep((long) (delay * 1000));
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return delay;
    }

    public static void main(String[] args) {
        List<String> servers = List.of("Server1", "Server2", "Server3");
        LeastResponseTime leastResponseTimeLB = new LeastResponseTime(servers);

        for (int i = 0; i < 6; i++) {
            String server = leastResponseTimeLB.getNextServer();
            System.out.println("Request " + (i + 1) + " -> " + server);
            double responseTime = simulateResponseTime(server);
            leastResponseTimeLB.updateResponseTime(server, responseTime);
            System.out.println("Response Time: " + String.format("%.2f", responseTime) + "s");
        }

    }
}

Component diagram for Load Balancing Algorithms showing each building block and its responsibility — Key components of Load Balancing Algorithms

Python

python

import time
import random

class LeastResponseTime:
    def __init__(self, servers):
        self.servers = servers
        self.response_times = [0] * len(servers)

    def get_next_server(self):
        min_response_time = min(self.response_times)
        min_index = self.response_times.index(min_response_time)
        return self.servers[min_index]

    def update_response_time(self, server, response_time):
        index = self.servers.index(server)
        self.response_times[index] = response_time

# Simulated server response time function
def simulate_response_time():
    # Simulating response time with random delay
    delay = random.uniform(0.1, 1.0)
    time.sleep(delay)
    return delay

# Example usage
servers = ["Server1", "Server2", "Server3"]
load_balancer = LeastResponseTime(servers)

for i in range(6):
    server = load_balancer.get_next_server()
    print(f"Request {i + 1} -> {server}")
    response_time = simulate_response_time()
    load_balancer.update_response_time(server, response_time)
    print(f"Response Time: {response_time:.2f}s")

IP Hash

Hashes the client's IP address to determine which server handles the request. Ensures the same client always reaches the same server (session affinity).

Time Complexity: O(1)
Space Complexity: O(N)
Best For: Stateful applications requiring session affinity without cookies

Interview preparation checklist for Load Balancing Algorithms with key points to mention and mistakes to avoid — Interview tips for Load Balancing Algorithms

Java

java

import java.util.List;

public class IPHash {
    private List<String> servers;

    public IPHash(List<String> servers) {
        this.servers = servers;
    }

    public String getNextServer(String clientIp) {
        int hash = clientIp.hashCode();
        int serverIndex = Math.abs(hash) % servers.size();
        return servers.get(serverIndex);
    }

    public static void main(String[] args) {
        List<String> servers = List.of("Server1", "Server2", "Server3");
        IPHash ipHash = new IPHash(servers);

        List<String> clientIps = List.of("192.168.0.1", "192.168.0.2", "192.168.0.3");
        for (String ip : clientIps) {
            System.out.println(ip + " is mapped to " + ipHash.getNextServer(ip));
        }
    }
}

Python

python

import hashlib

class IPHash():
    def __init__(self, servers):
        self.servers = servers

    def get_next_server(self, client_ip):
        hash_value = hashlib.md5(client_ip.encode()).hexdigest()
        index = int(hash_value, 16) % len(self.servers)
        return self.servers[index]

# Example usage
servers = ["Server1", "Server2", "Server3"]
load_balancer = IPHash(servers)

client_ips = ["192.168.0.1", "192.168.0.2", "192.168.0.3", "192.168.0.4"]
for ip in client_ips:
    server = load_balancer.get_next_server(ip)
    print(f"Client {ip} -> {server}")

Production Considerations

Health Checks: All algorithms should skip unhealthy servers. Add a health status flag to each server.
Connection Draining: During deployments, stop sending new requests to a server but let existing connections finish.
Thread Safety: In multi-threaded environments, protect shared state (current index, connection counts) with locks or atomic operations.
Monitoring: Track requests per server, error rates, and response times to validate your algorithm is distributing load correctly.

Decision guide for when to choose Load Balancing Algorithms and when alternative approaches are better — When to use Load Balancing Algorithms

Source

Code from System-Design-Overview by Ashish Pratap Singh. Explanations are original.

Original DosiBridge notes for Load Balancing Algorithms

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Load Balancing Algorithms is to connect the concept to a traffic-management problem where overload behavior matters more than the happy-path diagram. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

Practice Loop: For Load Balancing Algorithms, decide the practice loop before choosing tools; otherwise the design looks clean but fails when load changes.
Implementation Exercise: For Load Balancing Algorithms, decide the implementation exercise before choosing tools; otherwise the design looks clean but fails when balancing changes.
Review Checkpoint: For Load Balancing Algorithms, decide the review checkpoint before choosing tools; otherwise the design looks clean but fails when algorithms changes.
Failure Drill: For Load Balancing Algorithms, decide the failure drill before choosing tools; otherwise the design looks clean but fails when load changes.
Comparison Target: For Load Balancing Algorithms, decide the comparison target before choosing tools; otherwise the design looks clean but fails when balancing changes.

Production review question

If a team adopts Load Balancing Algorithms, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track load-algorithms health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Load Balancing Algorithms, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.

Sources

Source Code on GitHubcode

Load Balancing Algorithms

Quick answer: Load Balancing Algorithms

Related reading path

Load Balancing Algorithms — Code Lab

Algorithm Comparison

Round Robin

Java

Python

Weighted Round Robin

Java

Python

Least Connections

Java

Python

Least Response Time

Java

Python

IP Hash

Java

Python

Production Considerations

Source

Original DosiBridge notes for Load Balancing Algorithms

What makes this topic distinct

Production review question

Practical metric to track

Sources