The dual-write problem: what an outbox actually guarantees

Write the failure table before choosing a tool

Suppose an application creates an order in a database and publishes an order-created message. If it commits the order first, then crashes before publication, downstream systems never learn about an existing order. If it publishes first, then the database transaction fails, downstream systems act on an order that does not exist. Running the two operations concurrently adds more possible interleavings without creating atomicity. The failure is not a rare implementation accident. It follows directly from asking two independent durable systems to agree through separate operations.

I would first state which system is authoritative and what downstream readers may temporarily observe. For many workflows, the database owns the accepted business change and a delayed notification is tolerable, while a permanently missing notification is not. That leads naturally to durable publication intent. Other workflows require a different authority or a distributed transaction with participants that support it. Choosing an outbox should follow the required invariant, rather than a general belief that asynchronous messaging automatically makes a system more reliable or easier to operate.

Commit the decision and the intent together

An outbox places the business change and an event describing that change inside one local transaction. Either both commit or neither does. A separate relay later publishes committed outbox records. The relay can crash or lose acknowledgements without erasing the durable intent. This solves the specific gap between accepting a local decision and remembering that something must be published. It does not require the request handler to wait for the broker, although the application must tolerate the delay between the local commit and downstream visibility.

The event should represent a fact with a stable identity, not an instruction reconstructed from whatever the row happens to contain when the relay runs. In a hypothetical order workflow, include the order identifier, event identifier, event type, schema version and relevant committed revision. The exact payload depends on the consumers' contract. The SQL sketch assumes the supplied identifiers are validated and uniqueness is enforced. If the business operation is retried, its own idempotency boundary must prevent creating a second order and a second logically duplicate outbox event.

BEGIN;
WITH created AS (
  INSERT INTO orders (id, status, revision)
  VALUES ($1, 'accepted', 1)
  RETURNING id, revision
)
INSERT INTO outbox (id, aggregate_id, event_type, payload)
SELECT $2, id, 'OrderAccepted',
       jsonb_build_object('order_id', id, 'revision', revision)
FROM created;
COMMIT;

The relay has an acknowledgement problem of its own

After publishing an outbox record, a relay usually needs to record that publication succeeded. If it marks the record complete before the broker accepts it, a crash can lose the event. If it marks it complete afterwards, a crash between acceptance and the local update can publish it again. The outbox has not eliminated this uncertainty; it has moved it into a recoverable delivery stage. The normal response is to preserve the event identifier and require consumers to tolerate repeated delivery within a defined retention horizon.

A polling relay also needs a safe claim protocol. Workers can use short leases or database-supported row claiming, but a lease expiry does not prove the old worker has stopped executing. A delayed worker can still publish after another worker takes over. Therefore leasing improves scheduling and availability without replacing idempotency. I would track the oldest unpublished event, attempt count, last failure and lease owner separately. A relay that reports a high publication rate can still hide one permanently stuck aggregate, so completion and freshness need measurements tied to individual outstanding work.

Change-data capture changes the relay, not the business boundary

Debezium's outbox event router documents an event identifier and an aggregate identifier used as the emitted message key. Those fields support deduplication and partitioning decisions. Change-data capture can read committed outbox changes from the database log instead of repeatedly querying the table. I would view this as an implementation of the relay, not as proof that all downstream effects occur once. The business transaction still establishes the intent, while the connector, broker and consumer each have their own durable progress and recovery rules.

PostgreSQL's logical decoding documentation explicitly warns that a crash can cause recent changes to be sent again because a slot's persisted position may move back to its checkpointed value. That is a concrete reason to retain stable event identities across CDC delivery. Operationally, the connector also becomes a consumer with a retention obligation: if it falls behind, required log data must remain available or recovery needs another plan. I would monitor connector position and retained log alongside broker lag, because a healthy broker cannot compensate for changes that never leave the database.

References: [1] Debezium: Outbox Event Router[2] PostgreSQL: Logical Decoding Concepts

Ordering follows the aggregate's invariant

An order-accepted event and an order-cancelled event may need to reach a consumer in that order. Events for unrelated orders may not. Using the order identifier as a partition key can align transport order with this requirement, but the producer and relay must still preserve the intended sequence. Parallel relays that publish later revisions first can break the assumption before the broker sees the records. I would put an explicit aggregate revision in each event and make the consumer's response to duplicates, gaps and older revisions part of its contract.

For a hypothetical consumer that has applied revision 7, revision 7 again is a duplicate, revision 6 is stale, and revision 9 exposes a gap if every revision is required. A current-state projection may safely fetch the latest authoritative version instead of waiting for revision 8. A workflow that must execute every transition cannot make that substitution. This is why one universal outbox consumer template is insufficient. The transport can carry the same envelope for both consumers while their domain-specific recovery behaviour differs substantially.

Retention and observability determine whether it stays recoverable

Assume an illustrative workload produces 3,000 outbox events per second with an average stored size of 700 bytes. One hour of retained payload alone is about 7.56 gigabytes before indexes, transaction logs and replication. If the broker is unavailable for a day, the backlog is no longer a minor housekeeping concern. The database needs capacity for the promised outage horizon, and the application needs a policy for what happens as that capacity runs out. Durable intent is valuable only while the system can continue preserving it.

Cleanup should depend on verified relay progress and any replay requirement, not simply row age. Deleting an event because it is old can remove precisely the record that repeated delivery failures prevented from leaving the system. Conversely, retaining every published event in the transactional database forever may be unnecessary if an appropriate archive owns the historical contract. I would separate operational outbox retention from audit retention, name the owner of each, and test restoration. Restoring a database backup can reintroduce already published events, so recovery procedures need the same identity discipline as ordinary retries.

Sometimes the correct fix is fewer boundaries

The strongest counterargument is that an outbox adds a table, a relay, monitoring and eventual consistency to a workflow that used to be a straightforward request. That cost is real. If the supposedly separate consumers can reasonably share the same transactional database, a single transaction may be simpler. If the notification is disposable, a best-effort publish with an honest loss policy may suffice. I would not install an outbox merely to satisfy an architectural fashion. Its value is preserving an important publication obligation across failure.

When that obligation matters, test it directly. Stop the process after the business commit, interrupt the relay after broker acceptance, repeat an event, restore an old checkpoint and pause the connector until retention becomes relevant. Verify both that every committed intent remains discoverable and that repeated delivery does not repeat the protected business effect. The outbox's strongest promise is local and precise: accepted state changes cannot silently forget their corresponding publication intent. Everything beyond that boundary deserves equally explicit treatment instead of being smuggled into the phrase reliable messaging.