Why “unknown” should be a valid system output

The answer space is larger than yes and no

Consider an illustrative funding investigation. An application observes a transfer between two addresses, then loses visibility at an intermediary. Asked whether one person funded another, it can produce three materially different outcomes: evidence supports the proposition, evidence supports its negation, or the available evidence supports neither. A network timeout is a fourth event: the attempted investigation did not complete. Encoding all four as a boolean forces the caller to invent semantics. The visible simplicity belongs to the response shape; the complexity survives in decisions made downstream.

I would begin by writing down the proposition precisely. No observed transfer in a specified dataset during a specified interval is different from no transfer ever occurred. The first claim might be supported by a complete scan; the second may exceed the system's observation boundary. Abstention should attach to the question actually asked, not an indefinitely broader question. This matters for Wallet Tracer's stated interest in evidence and explicit uncertainty, but the example here describes a contract design, not unpublished tracing behaviour.

Abstention has an economic shape

A simple illustrative loss model makes the decision concrete. Let p be a calibrated probability that a proposition is true. Assign a cost of 100 units to a false positive, 5 units to a false negative, and 2 units to deferring the decision. A positive answer has expected loss 100 times (1 minus p); a negative answer has loss 5 times p; deferral costs 2. Among these three actions, negative is preferred below p = 0.4 and positive above p = 0.98. Between those thresholds, deferral has the smallest expected loss.

This does not establish a universal confidence threshold. It establishes which assumptions a threshold hides. The numbers are invented for the example, the probability must deserve its interpretation, and ties need a declared policy. If review costs 20 units instead, the defer region may disappear entirely for this loss structure. If a wrong action is irreversible, its cost may dwarf the inconvenience of waiting. I prefer making those choices legible to treating confidence above an arbitrary decimal as permission to act.

Separate the decision from execution failure

Rust's enums allow each outcome to carry different data. An outer Result can describe whether evaluation ran, while an inner decision distinguishes supported answers from abstention. A supported negative answer then has exactly the same structural dignity as a supported positive answer. It carries evidence references and a policy version. An abstention carries a reason that can guide recovery. This is more informative than Result<Option<bool>, Error>, even though that nested type can represent roughly the same number of states.

The illustrative type below deliberately avoids a bare confidence field. Some decisions are based on completeness constraints rather than probability, and a fabricated numeric score is worse than an honest categorical reason. If calibrated probabilities are available, they belong in a versioned assessment with a defined population and calibration method. The enum cannot prove that evidence is sufficient. It can ensure that a developer confronts the distinction before serializing a result or writing the next match statement. That modest guarantee is valuable precisely because it is specific.

enum Verdict<T> {
    Supported { value: T, evidence: Vec<String>, policy: String },
    Abstain { reason: Abstention, evidence: Vec<String> },
}
enum Abstention {
    MissingCoverage,
    ConflictingEvidence,
    OutsideModelScope,
}
enum EvaluationError { Unavailable, InvalidInput }
type Decision<T> = Result<Verdict<T>, EvaluationError>;

References: [2] The Rust Programming Language: Defining an Enum

Coverage is a metric, not a loophole

Selective classification studies the trade between making predictions on more inputs and reducing risk on the inputs accepted. That framing is useful beyond machine learning: measure both how often a system answers and how often its answers are wrong. A service that answers one unusually easy question perfectly has not solved the same problem as one that answers most questions. Conversely, a service that never abstains may conceal serious errors inside an impressive completion rate. Neither metric is interpretable without knowing which cases were accepted.

My proposed evaluation would preserve the original population, then report answered cases, abstained cases and reasons by relevant slice. In an illustrative batch of 1,000 investigations, 700 supported answers and 14 adjudicated errors imply 70 percent coverage and 2 percent error among answered cases, assuming all 700 received reliable adjudication. They do not imply that the other 300 are harmless. If difficult cases disproportionately belong to one class of users, aggregate improvement may simply move the unresolved work onto that group.

Distribution shift makes this accounting more important. A confidence threshold tuned on familiar inputs may accept unfamiliar ones for the wrong reasons. I would retain a separate out-of-scope path and evaluate new input populations explicitly, rather than interpreting an unchanged acceptance rate as evidence that calibration survived the change.

References: [1] Selective Classification for Deep Neural Networks — Geifman and El-Yaniv

A review queue changes the optimal policy

Deferral is often described as sending a case to a human, as though humans supplied an unlimited reliable fallback. A real review path has capacity, delay and error. If the illustrative system abstains on 300 cases per hour but reviewers resolve 100, the queue grows by 200 cases per hour while those rates persist. The two-unit deferral cost used earlier is now wrong because waiting time is changing. Abstention remains epistemically honest, but the surrounding product has made a promise it cannot sustain.

A workable contract therefore separates the reason for abstaining from the resolution offered to the user. Missing evidence might permit a targeted refresh; a scope boundary might require a different tool; conflicting records might justify a review task. Some cases should remain unresolved rather than circulate through retries. I would put a deadline and an accountable owner on escalations, and measure eventual resolution separately from initial answer quality. The decision engine should not silently become more confident merely because the review team is busy.

Unknown must survive composition

The dangerous conversion often happens a service later. A database column defaults to false, an analytics export drops nulls, or a UI renders every missing label as safe. The richer producer type then becomes decoration. I would test the complete path with an abstention fixture: serialization, storage, aggregation, display and any automated action. The crucial assertion is not that the word unknown appears somewhere. It is that no downstream component grants a permission or asserts a negative proposition from the absence of support.

Combining decisions also needs semantics. Two unknown assessments do not become a supported negative because neither found evidence. Two positive assessments based on the same underlying record are not independent corroboration. An all-checks-passed policy may abstain if one required check abstains, whereas a sufficient-single-proof policy may still conclude from another decisive source. Those are different logical operators. Give them names and test their truth tables, including conflicting supported results, rather than allowing generic boolean operators to select a policy accidentally.

Abstention is accountable restraint

The strongest objection is that an explicit unknown state can excuse a weak system. That objection is fair. A team can hide poor coverage behind a language of caution, or make the user perform the difficult work without improving anything. The answer is to make abstention inspectable: name the missing evidence, identify whether it can be obtained, and keep a history of whether attempted improvements reduce the relevant gap. An unexplained refusal can be as unhelpful as an unexplained assertion, even when it is less misleading.

My preference is a system that earns its supported answers and accounts for its refusals. That requires two contracts: a knowledge contract describing what the evidence permits, and an action contract describing what happens next. The first must not inflate confidence to satisfy the second. The second must not abandon the user simply because the first is uncertain. Once those responsibilities are separated, unknown becomes a useful outcome to engineer around rather than an embarrassing blank that someone eventually fills with a guess.