How Real-Time Messaging Works

How Real-Time Messaging Works

Real-time messaging means delivering data to clients within milliseconds of it being produced, without the client explicitly asking for it. This is fundamentally different from the request-response model of HTTP, and the choice of protocol has significant implications for scalability, reliability, and complexity.

The Three Approaches

HTTP Long Polling

The client sends an HTTP request, and the server holds the connection open until new data is available (or a timeout occurs). When data arrives, the server responds, the connection closes, and the client immediately opens a new one.

Pros: Works through all firewalls and proxies. Uses standard HTTP, so no special infrastructure is needed. Simple to implement.

Cons: Each poll cycle requires a new TCP connection (or at least a new HTTP request), adding overhead. The server must hold many open connections waiting for data. Not truly real-time — there is a small gap between the response and the next poll.

Use when: You need basic real-time behavior and cannot use WebSocket (legacy environments, restrictive firewalls).

Server-Sent Events (SSE)

The client opens a single HTTP connection, and the server sends events down that connection as they occur. The connection stays open indefinitely. SSE is built into the browser via the EventSource API.

Pros: Simple one-directional streaming over standard HTTP. Automatic reconnection built into the browser API. Works through most proxies and CDNs. Lightweight — no custom protocol negotiation.

Cons: Unidirectional (server to client only). Clients cannot send messages back over the same connection. Limited to text data (no binary). Some older proxies may buffer responses, breaking the streaming behavior.

Use when: You only need server-to-client updates (live scores, stock tickers, notification feeds). No need for client-to-server messaging on the same connection.

WebSocket

After an initial HTTP handshake (upgrade request), the connection is upgraded to a persistent, full-duplex TCP connection. Both client and server can send messages at any time.

Pros: Full-duplex communication. Very low overhead per message (2-6 byte frame header vs. hundreds of bytes for HTTP headers). Supports binary data. True real-time: no polling gaps.

Cons: Requires WebSocket-aware infrastructure (load balancers, proxies). Stateful connections complicate horizontal scaling (sticky sessions or connection-aware routing). Some corporate firewalls block WebSocket upgrade requests.

Use when: You need bidirectional real-time communication — chat, collaborative editing, multiplayer games, live trading.

Connection Management at Scale

The hard part of real-time messaging is not the protocol — it is managing millions of persistent connections.

Connection gateways: Dedicated servers that hold WebSocket connections. Each gateway handles 10,000-100,000 connections. Gateways are stateless in terms of business logic; they only maintain the mapping of "which user is connected to this server."

Connection migration: When you deploy a new version of the gateway, existing connections must be gracefully drained. Clients reconnect to a new gateway instance. During migration, messages must not be lost.

Heartbeats: Both client and server send periodic pings to detect dead connections. Without heartbeats, a disconnected client consumes server resources until a TCP timeout (which can take minutes).

Reconnection with gap-fill: When a client reconnects after a brief disconnect, it reports the last message ID it received. The server sends all messages since that ID, ensuring no messages are lost.

Message Ordering

Guaranteeing that all users see messages in the same order is harder than it sounds.

Per-channel sequence numbers: Assign a monotonically increasing sequence number to each message within a channel. All clients sort by this number. This guarantees total order within a channel.

Cross-channel ordering: Not usually necessary or feasible. Users see messages in their channel's order, and different channels can have independent sequences.

Conflict resolution: In collaborative editing scenarios where two users type simultaneously, you need algorithms like Operational Transformation (OT) or CRDTs (Conflict-free Replicated Data Types) to merge concurrent edits.

The Fanout Problem

When a message is posted to a channel with 50,000 members, delivering it to all online members is the core scalability challenge.

Fanout-on-write: When the message is written, immediately push it to all recipients. Fast for recipients, expensive for the sender (especially for channels with many members).

Fanout-on-read: Store the message once; recipients fetch it when they check the channel. Cheaper to write, but reads are slower and more complex.

Hybrid approach: Use fanout-on-write for small channels (most channels) and fanout-on-read for very large channels (announcements, public channels with thousands of members).

Comparison Table

Summary

Use long polling as a fallback, SSE for server-to-client streams, and WebSocket for bidirectional real-time communication. The real challenge is connection management at scale — gateways, heartbeats, reconnection, and message fanout. Per-channel sequence numbers solve ordering.

Original DosiBridge notes for How Real-Time Messaging Works

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study How Real-Time Messaging Works is to connect the concept to a production service where growth exposes the first weak assumption in the original design. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Practical Lesson: For How Real-Time Messaging Works, decide the practical lesson before choosing tools; otherwise the design looks clean but fails when how changes.
• Migration Story: For How Real-Time Messaging Works, decide the migration story before choosing tools; otherwise the design looks clean but fails when real changes.
• Production Symptom: For How Real-Time Messaging Works, decide the production symptom before choosing tools; otherwise the design looks clean but fails when time changes.
• Decision Checklist: For How Real-Time Messaging Works, decide the decision checklist before choosing tools; otherwise the design looks clean but fails when messaging changes.
• Rollback Plan: For How Real-Time Messaging Works, decide the rollback plan before choosing tools; otherwise the design looks clean but fails when works changes.

Production review question

If a team adopts How Real-Time Messaging Works, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track how-works health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For How Real-Time Messaging Works, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.

Sources

• Real-Time Messaging at Slack — Slack Engineering