Here’s a bug that’s notoriously hard to catch in production: you save an order to the database, then publish an order.created event to your message broker — and the broker crashes between those two steps. Your database has the order. Nobody got the event. Downstream services are none the wiser.

Or flip it: the event goes out, then your database write fails. Now you’re broadcasting a lie.

This is the dual-write problem, and it affects every system that tries to update two things atomically (a DB and a message broker) without a distributed transaction. The Outbox pattern solves it cleanly.

The Root Cause

The problem is that a database write and a message broker publish are two separate I/O operations. There’s no way to wrap them in a single atomic operation without heavy infrastructure like XA transactions — which are slow, complex, and not supported by most brokers.

// ❌ This looks fine, but it's broken
async function createOrder(data: OrderData) {
  const order = await db.orders.create(data); // Step 1
  await messageBroker.publish('order.created', order); // Step 2 — can fail independently
  return order;
}

If step 2 fails, you have a phantom order. If your process crashes between steps, same result.

The Outbox Pattern

The fix is elegant: instead of publishing directly to the broker, write the event to a special “outbox” table in the same database transaction as your business data. A separate process (the relay) reads from that table and publishes to the broker.

┌─────────────────────────────────────┐
│           Database Transaction       │
│  ┌──────────────┐  ┌─────────────┐  │
│  │  orders      │  │  outbox     │  │
│  │  (new row)   │  │  (new event)│  │
│  └──────────────┘  └─────────────┘  │
│         Atomic — both or neither    │
└─────────────────────────────────────┘
              ↓ (relay reads)
      ┌───────────────────┐
      │   Message Broker  │
      └───────────────────┘

Now the database write and the event creation are atomic. The relay handles delivery separately.

Implementation

Step 1: Create the outbox table

CREATE TABLE outbox_events (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  aggregate_type VARCHAR(100) NOT NULL,
  aggregate_id VARCHAR(100) NOT NULL,
  event_type VARCHAR(100) NOT NULL,
  payload JSONB NOT NULL,
  created_at TIMESTAMPTZ DEFAULT now(),
  published_at TIMESTAMPTZ,
  retry_count INT DEFAULT 0
);

Step 2: Write to the outbox in the same transaction

import { db } from './db';

async function createOrder(data: OrderData): Promise<Order> {
  return await db.transaction(async (trx) => {
    // Business write
    const order = await trx('orders').insert(data).returning('*')[0];

    // Outbox write — same transaction
    await trx('outbox_events').insert({
      aggregate_type: 'Order',
      aggregate_id: order.id,
      event_type: 'order.created',
      payload: JSON.stringify({
        orderId: order.id,
        customerId: order.customerId,
        items: order.items,
        total: order.total,
      }),
    });

    return order;
  });
}

Either both writes succeed, or neither does. The dual-write problem is gone.

Step 3: Build the relay

The relay polls the outbox table and publishes unpublished events:

import { messageBroker } from './broker';
import { db } from './db';

async function relayOutboxEvents(): Promise<void> {
  const events = await db('outbox_events')
    .whereNull('published_at')
    .orderBy('created_at', 'asc')
    .limit(100);

  for (const event of events) {
    try {
      await messageBroker.publish(event.event_type, JSON.parse(event.payload));

      await db('outbox_events').where({ id: event.id }).update({
        published_at: new Date(),
      });
    } catch (err) {
      await db('outbox_events').where({ id: event.id }).update({
        retry_count: event.retry_count + 1,
      });
      console.error(`Failed to publish event ${event.id}:`, err);
    }
  }
}

// Run every second
setInterval(relayOutboxEvents, 1000);

Handling duplicates

The relay might publish the same event more than once (e.g., if it crashes after publishing but before marking the event as done). Consumers must be idempotent:

// Consumer side — deduplicate using event id
const processedEvents = new Set<string>();

messageBroker.subscribe('order.created', async (event) => {
  if (processedEvents.has(event.id)) {
    return; // Already handled
  }

  await handleOrderCreated(event);
  processedEvents.add(event.id);
});

In production, store processed event IDs in a database (not in-memory) for durability.

Alternative: CDC (Change Data Capture)

Instead of polling, you can use Change Data Capture tools like Debezium to stream database changes directly to Kafka. This is more efficient for high-throughput systems:

# Debezium connector config (simplified)
name: orders-connector
config:
  connector.class: io.debezium.connector.postgresql.PostgresConnector
  database.hostname: postgres
  database.dbname: mydb
  table.include.list: public.outbox_events
  transforms: outbox
  transforms.outbox.type: io.debezium.transforms.outbox.EventRouter

CDC reads the database’s write-ahead log (WAL) directly, making it zero-overhead for the application.

When to Use the Outbox Pattern

Use it when:

  • You publish events after database writes and need guaranteed delivery
  • You can’t afford to lose events (financial, order, or audit systems)
  • You’re building event-driven microservices

You might skip it when:

  • You’re okay with at-most-once delivery
  • You control both sides of the transaction (same DB for both services)
  • The operation is idempotent and you can retry safely

Key Takeaways

  • The dual-write problem is real and will bite you in production
  • The Outbox pattern uses an atomic DB write to guarantee event creation
  • A relay (polling or CDC) handles the actual broker publish
  • Consumers must be idempotent because at-least-once delivery is the contract
  • Debezium + Kafka is the production-grade version of this pattern