Exactly once needs a boundary

Start with the ambiguous acknowledgement

Imagine a worker that receives a command, updates a database and acknowledges the message. It crashes after the update commits but before the acknowledgement reaches the broker. From the broker's perspective, the message is unfinished. From the database's perspective, its effect already exists. Redelivering the message is the correct recovery behaviour for the broker, yet executing the update again may be wrong for the application. Neither component has failed its local contract. The problem is that their contracts do not share an atomic decision.

Moving the acknowledgement before the update changes the failure, rather than removing it. A crash in the gap now loses work. This two-step example is enough to reject an unqualified claim that the message is processed exactly once. The worker may execute its handler repeatedly. What the user usually needs is that a particular durable business effect appears once despite those executions. I would write that requirement explicitly, including the storage system in which the effect is observed and what a caller should receive while the outcome remains uncertain.

Put the receipt beside the effect

If the deduplication record and business update live in the same transactional database, the worker can make one atomic decision about both. A unique key identifies the operation within a consumer's namespace. Insert that receipt and apply the update in the same transaction; on conflict, return the previously recorded outcome instead of applying the effect again. PostgreSQL's constraint documentation supplies the relevant uniqueness mechanism. The broader correctness argument depends on transaction placement: a unique identifier stored in an unrelated cache does not protect a database mutation.

For a hypothetical credit operation, let the initial balance be 40 and the requested increment be 7. Two concurrent deliveries should leave 47, with one receipt, rather than 54. The example requires UNIQUE (consumer, operation_id) and conditions receipt insertion on an existing, locked account. The caller must classify a zero-row result and roll back a missing-account request. Production also needs a stored request fingerprint and result, intentionally omitted here: a repeated identifier must match the original account, amount and operation type before its saved outcome is returned. Otherwise deduplication can silently substitute a different request's answer.

BEGIN;
-- Required: UNIQUE (consumer, operation_id).
WITH target AS (
  SELECT account_id FROM balances
  WHERE account_id = $3 FOR UPDATE
), accepted AS (
  INSERT INTO processed_operations (consumer, operation_id)
  SELECT 'credit-balance', $1 FROM target
  ON CONFLICT (consumer, operation_id) DO NOTHING
  RETURNING operation_id
)
UPDATE balances SET amount = amount + $2
WHERE account_id = $3
  AND EXISTS (SELECT 1 FROM accepted);
-- Caller distinguishes duplicate from missing account.
-- Missing account: ROLLBACK and return an error, not success.
-- Production also persists/validates request fingerprint and result.
COMMIT;

References: [1] PostgreSQL: Constraints

A broker transaction has a useful, specific scope

Kafka's design documentation explains a bounded exactly-once construction: consumed offsets and output records can commit together in a Kafka transaction, with consumers reading committed results. That protects a read-process-write workflow inside the relevant Kafka boundary. The same documentation distinguishes external destinations, which require cooperation. I take that distinction seriously. If the handler also calls a payment service or sends an email, the broker transaction does not reach backward through the network and make that independent service's action atomic with the offsets.

The practical response is to enumerate sinks rather than argue about the slogan. A pipeline might have transactional output to one topic, idempotent writes to a database, and best-effort telemetry. Each sink has a different recovery story. The system can still be well designed, but its guarantee is a composition of those stories. I would document the weakest externally relevant effect and make it visible during failure drills. A perfect topic history does not compensate for duplicated fulfilment, and a harmless repeated metric should not force every operation into an expensive coordination protocol.

References: [2] Apache Kafka 4.1 design: Message delivery semantics

External effects need an identity the receiver understands

For a remote service, the caller's local receipt is insufficient. The receiver needs an idempotency contract, or the caller needs a reconciliation method that can identify the prior result. Stripe's API documentation describes idempotency keys, parameter comparison and a retention policy for stored results. Those are concrete properties, not a universal guarantee that all remote effects can be retried forever. A client should preserve the same operation identifier across an ambiguous timeout and understand when the receiver may have forgotten that identifier.

Suppose an invented supplier accepts a shipment request but its response is lost. Generating a new key on every retry defeats idempotency because the supplier sees new operations. Reusing the original key is appropriate only while the original request still represents the same business intent. If the delivery address changes, the workflow needs a new explicit action or a supported amendment. I would store the external request identity before sending, record the receiver's identifier when available, and expose an unresolved state when the remote outcome cannot yet be determined safely.

References: [3] Stripe API: Idempotent requests

Retention is part of the guarantee

Deduplication state has a lifetime, and that lifetime defines a failure boundary. Assume a hypothetical system allows message replay for thirty days but deletes receipts after seven. A delivery from day eight can legitimately encounter no remembered receipt and repeat an old effect. The system has not provided thirty-day duplicate protection, whatever the happy-path handler does. The retention policy must cover the permitted replay horizon, delayed retries, operational restores and any clock uncertainty used to expire entries. A storage cleanup job can therefore change application semantics.

There is also a scale calculation worth making explicit. At an assumed 2,000 unique operations per second, thirty days contains about 5.184 billion operation identities. Even 32 bytes per identity is roughly 166 gigabytes before indexes, row overhead and replication. This is not an argument against deduplication. It is a reason to choose a representation and retention boundary deliberately. A monotonic per-entity sequence can sometimes replace many individual receipts, but only if gaps, parallel delivery and old operations have defined handling. Compressing state by forgetting uncertainty does not preserve the original guarantee.

Exactly once does not decide whether the effect is right

A single committed update can still be the wrong update. The receipt key may identify the wrong business operation, the amount may be calculated under an obsolete rule, or two different event identifiers may describe the same underlying action. Transport deduplication only recognises the identities it receives. I would separate delivery identity, producer event identity and business operation identity in the model, then choose which one guards each effect. This distinction is especially important when an upstream system emits both an object-created event and a later status event for the same workflow.

The strongest counterargument is that the implementation detail should stay hidden behind a trustworthy platform. For many ordinary applications, that is a good goal. Developers should use managed transactional primitives instead of recreating them. However, abstraction does not eliminate scope. Someone still has to determine whether the promised effect is inside the platform's boundary, whether external calls participate, and how long recovery remains supported. A clear platform guarantee can simplify that reasoning considerably; a label without those conditions can only postpone it until the first ambiguous failure.

Test the boundary as a state machine

A useful test suite crashes or disconnects the worker at every boundary between durable steps. Test before the effect, during commit, after commit but before acknowledgement, and after acknowledgement is attempted but its result is unknown. Run two workers with the same identity, restore an older checkpoint, and retry with a conflicting payload. The expected result should be described in terms of durable state and observable outcome, not the number of times the handler function runs. Repeated execution can be perfectly compatible with a once-only effect.

My preferred design note ends with a sentence that can be falsified: for each valid operation identifier retained within this horizon, this database transaction applies this effect at most once, and successful retry eventually reveals the same outcome under these availability assumptions. At-most-once effect plus a recovery path addresses both safety and useful progress. Anything outside that statement needs another protocol or an explicit limitation. That is a smaller promise than exactly once everywhere, but it is a promise an engineer can inspect, test and explain.