Correlation IDs

A Correlation ID is a unique identifier attached to a request at the system's entry point and propagated through every service in the call chain. When a user clicks a button and that action flows through an API gateway, three microservices, two message queues, and a database, the Correlation ID ties all related logs, metrics, and traces together. Without it, debugging a failure in a 20-service architecture means searching through millions of unrelated log lines. With it, a single query retrieves the complete story of any request.

Why This Matters

In a monolithic application, a stack trace tells you exactly what happened. In a distributed system, a single user request might traverse 10 services, 3 message queues, and 2 databases. When something fails, you need to reconstruct the entire journey. Correlation IDs make this possible by giving every request a unique identity that persists across all service boundaries. Engineers can query their log aggregation system with a single ID and see every log line, from every service, in chronological order. This transforms incident response from hours of guesswork to minutes of targeted investigation. The W3C Trace Context specification (traceparent header) standardized this across the industry, and OpenTelemetry builds distributed tracing directly on top of these identifiers.

The Building Blocks

• ID Generation: Creating unique identifiers at the entry point — UUIDs, ULIDs, or W3C trace IDs — that are statistically guaranteed not to collide across billions of requests
• Header Propagation: Passing the correlation ID through HTTP headers (traceparent, X-Request-ID), gRPC metadata, and message queue headers at every service boundary
• Context Injection: Automatically adding the correlation ID to every log line, metric label, and trace span within each service using middleware or frameworks
• Async Propagation: Maintaining correlation across asynchronous boundaries — message queues, event streams, and background jobs — where HTTP headers do not naturally flow
• Correlation Lookup: Using the ID to query across telemetry systems, pulling related logs from Elasticsearch, traces from Jaeger, and metrics from Prometheus in a unified view

Under the Hood

Correlation IDs are implemented through three mechanisms: generation, propagation, and injection. At the system boundary (API gateway, load balancer), a unique ID is generated — typically a 128-bit value encoded as a 32-character hex string. The W3C traceparent header format is version-traceId-parentSpanId-flags (e.g., 00-4bf92f3577b86cd9f7e56fc4e8f7c9d2-00f067aa0ba902b7-01), carrying both the correlation ID and span hierarchy.

Propagation requires every service to extract the incoming correlation ID and attach it to all outbound requests. HTTP middleware extracts the traceparent header, stores it in a request-scoped context (ThreadLocal, AsyncLocal, Go context), and outbound HTTP clients, gRPC interceptors, and message queue producers read from that context. The challenge is async propagation: when a service publishes a message to Kafka, the correlation ID must be embedded in the message headers so the consumer can continue the trace.

Injection ensures the correlation ID appears in every telemetry signal. Logging frameworks read from the request context to automatically include the trace ID in every log line. Metric libraries tag measurements with the trace ID for exemplar linking. In practice, OpenTelemetry handles all three mechanisms — it generates trace IDs, propagates context through its SDK, and injects IDs into logs and metrics. The result is that a single ID query can reconstruct the complete distributed execution of any request.

How Companies Actually Do This

Amazon Uses X-Amzn-Trace-Id headers generated by Application Load Balancers and propagated through all AWS services, enabling X-Ray distributed tracing across Lambda, ECS, and EC2

Uber Propagates Jaeger trace IDs (uber-trace-id header) through every microservice and Kafka message, enabling engineers to trace a ride request from the mobile app through dispatch to driver notification

Shopify Attaches request IDs at their edge proxy and propagates them through all backend services, enabling support engineers to trace any merchant's failed checkout through their entire platform

Common Pitfalls

1. Generating new correlation IDs at each service boundary instead of propagating the existing one — destroys the ability to correlate events across the full request path
2. Not propagating correlation IDs across async boundaries (message queues, event streams) — the trace breaks at the most interesting and failure-prone boundaries
3. Using sequential or predictable IDs that leak information about request volume or could be used to forge trace membership in multi-tenant systems

Interview Questions Worth Practicing

1. How would you ensure correlation IDs propagate correctly through both synchronous HTTP calls and asynchronous message queues?
2. What is the W3C Trace Context standard and why did the industry standardize on a single correlation ID format?
3. How do correlation IDs relate to distributed tracing, and when would you need full tracing versus simple correlation IDs?

The Tradeoffs

• Simplicity vs. Richness: A single request ID is simple but limited; W3C traceparent with span IDs enables full parent-child trace reconstruction at the cost of complexity
• Automatic vs. Manual: Auto-instrumentation propagates IDs with zero developer effort but may miss custom protocols; manual instrumentation catches everything but requires discipline
• Overhead vs. Debuggability: Propagating context through every call adds minimal latency but the debugging and incident response benefits are transformational

How to Explain This in an Interview

Here is how I would explain Correlation IDs in a system design interview:

Correlation IDs are unique identifiers attached to requests at the system entry point and propagated through every service in the call chain. They are the foundation of distributed tracing. When a user request flows through 10 services, the correlation ID lets you query all related logs, metrics, and traces with a single identifier. I follow the W3C Trace Context standard using the traceparent header. The critical implementation detail is propagation — the ID must flow through HTTP headers, gRPC metadata, and message queue headers. OpenTelemetry handles this automatically. The most common mistake is generating new IDs at each service instead of propagating the original, which fragments the trace.

Related Topics

• Distributed Tracing
• Observability
• Logging
• Service Mesh

Original DosiBridge notes for Correlation IDs

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Correlation IDs is to connect the concept to a production service where growth exposes the first weak assumption in the original design. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Membership: For Correlation IDs, decide the membership before choosing tools; otherwise the design looks clean but fails when correlation changes.
• Clock Drift: For Correlation IDs, decide the clock drift before choosing tools; otherwise the design looks clean but fails when ids changes.
• Retry Semantics: For Correlation IDs, decide the retry semantics before choosing tools; otherwise the design looks clean but fails when correlation changes.
• Partition Behavior: For Correlation IDs, decide the partition behavior before choosing tools; otherwise the design looks clean but fails when ids changes.
• Reconciliation: For Correlation IDs, decide the reconciliation before choosing tools; otherwise the design looks clean but fails when correlation changes.

Production review question

If a team adopts Correlation IDs, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track correlation-ids health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Correlation IDs, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.