Publish-Subscribe Pattern

Pub/Sub (publish-subscribe) is a messaging pattern where publishers send messages to named topics and subscribers receive messages from topics they care about. Publishers and subscribers are fully decoupled — neither knows about the other. It is the backbone of event-driven architectures, powering everything from order processing pipelines to real-time notification systems at Google, LinkedIn, and Uber.

The Problem Publish-Subscribe Pattern Solves

Pub/Sub is the foundation of event-driven architectures. It enables microservices to communicate asynchronously, decouples producers from consumers, and handles traffic spikes by buffering messages.

How It Works Under the Hood

Pub/Sub (Publish-Subscribe) is a messaging pattern where publishers send messages to topics without knowing who will receive them, and subscribers receive messages from topics they are interested in. The publisher and subscriber are completely decoupled — they do not need to know about each other.

A publisher sends a message to a topic (e.g., 'order-placed'). The message broker (Kafka, RabbitMQ, Google Pub/Sub) stores the message and delivers it to all subscribers of that topic. Subscribers process messages asynchronously at their own pace.

This decouples the ordering service (publisher) from the payment service, inventory service, and notification service (subscribers). If the notification service is down, messages queue up and are processed when it recovers.

The Mental Model

• Topics: Named channels for messages. Publishers write to topics; subscribers read from topics.
• Decoupling: Publishers do not know about subscribers, and vice versa. You can add new subscribers without changing publishers.
• Fan-out: One message can be delivered to many subscribers simultaneously.
• At-least-once delivery: Most Pub/Sub systems guarantee each message is delivered at least once. Consumers must be idempotent.
• Ordering: Some systems guarantee ordering within a partition/topic (Kafka). Others do not (SNS).

Real Systems That Depend on This

Google Cloud Pub/Sub handles billions of messages per day for services like Gmail, YouTube, and Google Maps.

Apache Kafka is the most popular Pub/Sub system, used by LinkedIn, Netflix, and Uber for real-time event streaming.

AWS SNS + SQS is a common pattern: SNS (Pub/Sub) fans out to SQS queues (per subscriber), providing reliable async processing.

Where This Shows Up in Interviews

1. What is Pub/Sub and how does it differ from message queues?
2. When would you use Pub/Sub vs direct API calls?
3. How do you ensure messages are not lost?
4. What is the difference between fan-out and fan-in?

Tradeoffs

• Decoupling vs Debugging: Pub/Sub makes it hard to trace a request end-to-end.
• At-least-once vs Exactly-once: At-least-once is simpler; exactly-once requires idempotency or deduplication.
• Ordering vs Throughput: Strict ordering limits parallelism. Most systems order within a partition.

Watch Out For

1. Not making consumers idempotent — duplicate messages cause duplicate processing
2. Not monitoring consumer lag — a slow consumer falls behind and never catches up
3. Using Pub/Sub for synchronous request-response — use APIs instead

How to Explain This in an Interview

Here is how I would explain Publish-Subscribe Pattern in a system design interview:

Pub/Sub decouples producers from consumers through topics. A publisher writes an event to a topic — say 'order-placed'. Every service subscribed to that topic gets a copy: payment service charges the card, inventory service reserves stock, notification service sends a confirmation email. None of these services know about each other. The key difference from a message queue that interviewers test: in a queue, each message goes to ONE consumer (competing consumers pattern). In pub/sub, each message fans out to ALL subscribers. The critical implementation detail: consumers must be idempotent because most systems guarantee at-least-once delivery, meaning a consumer may process the same message twice during retries or rebalances.

Go Deeper

• Message Queues — start here if this is new to you
• Event-Driven vs Request-Driven
• Change Data Capture
• Sync vs Async

Original DosiBridge notes for Publish-Subscribe Pattern

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Publish-Subscribe Pattern is to connect the concept to an asynchronous workflow where replay, ordering, and poison messages shape the architecture. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• SLO: For Publish-Subscribe Pattern, decide the SLO before choosing tools; otherwise the design looks clean but fails when pub changes.
• Capacity Headroom: For Publish-Subscribe Pattern, decide the capacity headroom before choosing tools; otherwise the design looks clean but fails when sub changes.
• Failure Mode: For Publish-Subscribe Pattern, decide the failure mode before choosing tools; otherwise the design looks clean but fails when pub changes.
• Degradation Path: For Publish-Subscribe Pattern, decide the degradation path before choosing tools; otherwise the design looks clean but fails when sub changes.
• Operator Signal: For Publish-Subscribe Pattern, decide the operator signal before choosing tools; otherwise the design looks clean but fails when pub changes.

Distinct angle: subscriptions as independent reactions

Pub/sub is about independent reactions to a published fact. The topic owner defines the meaning of an event, while subscribers decide whether to send email, update analytics, invalidate cache, or start fulfillment. A strong design discusses schema evolution and replay so subscribers can change safely without forcing the publisher to know every downstream workflow.

Production review question

If a team adopts Publish-Subscribe Pattern, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track pub-sub health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Publish-Subscribe Pattern, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.