io_uring and buffer ownership

Submission transfers a temporary permission

When an application submits a read into a buffer, the kernel may write into that memory after the submission call returns. A submitted write similarly requires the source bytes to remain valid for the operation's lifetime. The io_uring manual distinguishes submission metadata from data buffers whose lifetime extends through completion. I would represent this as a temporary transfer of access permission: the application owns the allocation, but cannot freely mutate, free or reuse it while the operation still relies on it.

That distinction is easy to lose in an asynchronous wrapper. A caller drops the future, the wrapper drops a vector, and the operation is still outstanding. Memory ownership in the programming language must agree with the kernel's longer-lived access. A correct abstraction retains the allocation until the relevant completion boundary, even if nobody wants the result anymore. The difficulty is not specific to one syntax for asynchronous functions. It comes from composing a local lifetime with an external agent that continues using the memory independently.

References: [1] liburing: io_uring(7)

Give every outstanding operation an identity

Completions need not arrive in submission order. The user_data field can connect a completion to its operation, but the application must define a safe identity scheme. A pointer into a request table can work only while that entry remains valid and cannot be confused with a later occupant. I would use an operation identifier or a slot plus generation and maintain an outstanding-operation table. The completion handler then validates the identity before changing ownership, rather than assuming the next event belongs to the oldest request.

The generation rule matters during reuse. If slot seven is recycled while an old completion can still refer to it, a late event may free or modify the new operation's buffer. Increasing a generation helps detect that mistake, provided wraparound cannot recreate a live identity. Better still, do not recycle the slot until the protocol permits it. Identity checking is defence against stale events, not permission to release storage early. The table should make all outstanding references and terminal conditions inspectable during debugging and shutdown.

One-shot I/O has a small useful state machine

For an ordinary one-shot read, a buffer can move from free to prepared to submitted and finally completed before returning to free. The illustrative pseudocode assumes that the submission has been accepted and that each operation produces its normal completion. It deliberately excludes multishot requests, skipped-success completions and zero-copy send notifications. Those modes change the state machine. I would start with the narrowest operation set the application needs, because a universal completion handler can conceal incompatible ownership rules behind one generic done flag.

The result value must be interpreted before the buffer becomes useful to the caller. A read may complete with fewer bytes than requested or with an error. The valid initialized range follows the completion result, not the buffer's capacity. Returning a full-sized slice after a short read can expose stale or uninitialized data depending on the implementation. A wrapper should preserve this distinction in its result type. Completion establishes that the operation finished; it does not imply that the requested application-level record is complete or valid.

submit_read(slot):
  require slot.state == Prepared
  operation = fresh_identity(slot)
  retain(slot.buffer, operation)
  accepted_submit(operation, slot.buffer)
  slot.state = Submitted

on_terminal_read_completion(operation, result):
  slot = outstanding.remove_exact(operation)
  require slot.state == Submitted
  if result >= 0: deliver_initialized_prefix(slot.buffer, result)
  else: deliver_error(result)
  release_after_consumer_finishes(slot.buffer)

Cancellation has two identities to account for

A cancellation request is itself an operation with a result, while the target I/O has its own completion. The cancellation manual describes cases where the target is cancelled, not found or already progressing. I would track both identities and make buffer release depend on the target's required terminal evidence. A generic rule that frees the buffer when the cancellation request completes is too weak across these outcomes. It confuses the control request's result with the lifetime of the operation whose memory is still at issue.

There is also a race with normal completion. The target may finish successfully before cancellation reaches it. The application must handle that outcome once, even if the caller has already abandoned the result. A useful design separates delivering a result to the caller from reclaiming operation resources. The former can be suppressed after cancellation; the latter must still occur correctly. During shutdown, continue draining or otherwise resolving outstanding operations according to the documented interface instead of assuming that dropping the event loop ends every external access immediately.

References: [2] liburing: io_uring_prep_cancel(3)

Zero-copy send has a distinct release signal

The zero-copy send interface demonstrates why a result and a release signal cannot always share one event. Its documentation describes a send-result completion that, when marked with the MORE flag, is followed by a notification identifying when the associated memory can be reused. I would encode that additional state explicitly. Observing a successful send result does not then grant immediate permission to overwrite the source buffer. The second event is about memory lifetime, which is a different fact from the operation's reported send result.

This extra retention can affect the economics of avoiding copies. If notifications arrive slowly, the application needs more buffers or must apply backpressure. A copy-based path might release its application buffer earlier even while consuming additional bandwidth. The correct comparison includes retained memory and the latency of buffer availability, not merely bytes copied. I would also retain a fallback path for unsupported operation features or environments. Feature detection and operation-specific documentation belong to correctness, since io_uring capabilities depend on the kernel and configured interface.

References: [3] liburing: io_uring_prep_send_zc(3)

A bounded pool makes pressure observable

Assume an illustrative pool of 256 buffers, each with 16 KiB of payload capacity. The payload allocation is four MiB, before metadata and alignment. When every buffer is outstanding, the application must wait, reject work or use another explicitly budgeted resource. Allocating an unbounded replacement buffer hides backpressure until memory becomes the limiting mechanism. I would expose free, prepared, submitted and awaiting-release counts so the operator can distinguish a busy device from a completion-handling bug or slow downstream consumer.

The pool's ownership categories should sum to its total capacity. That simple invariant catches double returns and leaked slots more reliably than a free-list length alone. Submission failures require particular care: a batch may have some requests accepted and others still owned locally, depending on the API path. The implementation must classify each operation using actual submission results. Treating a partially accepted batch as entirely failed can reclaim buffers still in use; treating it as entirely submitted can strand buffers whose operations will never produce the expected completion.

The abstraction should make unsafe reuse difficult

The strongest counterargument is complexity. A straightforward blocking operation can keep a buffer's lifetime inside one call, which is easier to reason about and may be fast enough. I would adopt a ring-based completion engine when measured concurrency or batching benefits justify the larger state space. The public wrapper should expose owned buffers or borrow rules that survive cancellation, and its internal tests should exercise reordered completions, short results, submission failure and abandoned callers. A happy-path throughput test proves very little about these lifetime transitions.

A useful review walks one buffer through every terminal path and asks who can access it at each point. The answer should remain clear even when the result is unwanted or an error occurs. io_uring offers efficient ways to submit and receive work, but efficiency does not remove ownership. It makes the interval between request and completion more explicit. Designing around that interval produces an abstraction that can exploit the interface without depending on fortunate timing to keep its memory valid.

For diagnostics, retain the operation kind with its identity. A completion flag has meaning within an operation's documented protocol, and a generic event handler should not infer that meaning from the buffer alone. This small amount of metadata makes special modes reviewable instead of letting them silently inherit the one-shot path's assumptions.