Messaging Articles

Distributed Transactions Part 4: The Transactional Outbox

Part 4: The Transactional Outbox Mental Model > Connecting isolated components into a resilient, scalable, and observable distributed web. The Dual Write Problem occurs when you update your DB and then try to send an eve…

Apr 20, 20266 min read

#messaging

PlaybookKafka Production Playbook

Exactly-Once Semantics: The Mathematical Impossibility

Exactly-Once: The Mathematical Impossibility Mental Model > Connecting isolated components into a resilient, scalable, and observable distributed web. Exactly-once processing is impossible in a distributed system because…

Apr 20, 20266 min read

MessagingIntermediate

Kafka Consumer Groups Explained: Scaling Your Message Consumption

Kafka Consumer Groups Explained Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. In a production environment, a single consumer is often not enough to handle the volume o…

Deep DiveKafka & Event-Driven Mastery

Kafka Consumer Rebalancing: The Senior Engineer's Playbook

Kafka Consumer Rebalancing: Surviving the Storm Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. Consumer Group Rebalancing is one of the most common causes of latency sp…

PlaybookMessaging & Event-Driven Mastery

#java#messaging#performance

Kafka Internals: Zero-Copy, Log Storage, and High Throughput

Kafka Internals: The Secret to 10M+ Messages/Sec Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. Apache Kafka is often described as a distributed streaming platform, but…

Apr 20, 20268 min read

#ai-ml#java#messaging

Kafka Internals: The Storage Layer, Zero-Copy, and High Throughput

Kafka Internals: The Quest for 10M msg/sec Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. Apache Kafka is often described as a distributed streaming platform, but its h…

#ai-ml#java#messaging

Kafka Partition Skew Management

Kafka Partition Skew Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. If your partition key is poorly chosen, one consumer will do 90% of the work. We use load-based reba…

Apr 20, 20266 min read

#java#messaging#performance

MessagingBeginner

Kafka Rebalance Storms: Solving the 'Stop-the-World' Problem

Kafka Rebalance Storms: Mastering Consumer Group Stability Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. In a high-scale Kafka deployment, a Consumer Group Rebalance i…

Retry Queues vs. DLQ: Architecting Resilient Message Consumers

Retry Queues vs. DLQ: Beyond Simple Retries Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. In a high-scale messaging system, failures are not a matter of 'if', but 'whe…

ComparisonMessaging & Event-Driven Mastery

RabbitMQ Pitfalls: Managing Unacked Messages and Thundering Herds

RabbitMQ Pitfalls: Avoiding Messaging Meltdowns Mental Model > Connecting isolated components into a resilient, scalable, and observable distributed web. RabbitMQ is a robust broker, but misconfigurations can lead to sta…

#java#messaging#performance

RabbitMQ Internals: A Deep Dive into High-Performance Messaging

RabbitMQ Internals: Behind the Scenes Mental Model > Connecting isolated components into a resilient, scalable, and observable distributed web. RabbitMQ is one of the most reliable and widely used message brokers. Built…

RabbitMQ Quorum Queues: Modern High Availability with Raft

RabbitMQ Quorum Queues: A New Era of Reliability Mental Model > An immutable, high-speed transaction log used to decouple producers from consumers. For years, RabbitMQ relied on Classic Mirrored Queues for high availabil…

#ai-ml#messaging#system-design

The Transactional Outbox Pattern: Reliability in Microservices

The Transactional Outbox Pattern Mental Model > Connecting isolated components into a resilient, scalable, and observable distributed web. In a microservice, you often need to save data to a database (e.g., Order) and se…

Deep DiveBackend Systems Mastery

PlaybookMessaging & Event-Driven Mastery

Kafka Consumer Lag: Causes, Diagnosis, and Production Fixes

Kafka consumer lag is not a root cause. It is a symptom. Lag means consumers are not keeping up with producers for one or more partitions. The fix depends on why: slow processing, too few consumers, partition skew, rebal…

Apr 7, 20265 min read

#messaging

ComparisonMessaging & Event-Driven Mastery

Kafka Exactly-Once Semantics: Myth vs Production Reality

Kafka 0.11 introduced exactly-once semantics (EOS), and every architecture diagram since then has confidently placed a checkbox next to "exactly once delivery." In practice, most teams deploying Kafka with EOS still see…

Apr 20, 202511 min read

#databases#java#messaging

MessagingIntermediate

SQS vs Kafka vs EventBridge: Choosing the Right Messaging System on AWS

Every AWS backend team eventually faces the same decision: you need asynchronous messaging. SQS is right there in the console. Your architect says you need Kafka. Someone from DevOps mentions EventBridge. Each option has…

Apr 2, 202512 min read

#aws#messaging

Kafka Internals Deep Dive: Partitions, Offsets, and Consumer Groups

Apache Kafka is the de facto standard for event streaming in distributed systems, but most developers treat it as a black box — a durable message queue with a fancy name. Understanding Kafka's internals unlocks its true…

Jan 15, 202511 min read