Floating-point is an error budget

Representation is a contract about precision

A binary floating-point value represents one member of a finite set, with spacing that depends on magnitude. Many simple decimal fractions are not represented exactly. That fact does not make floating point defective; it means arithmetic results must be interpreted under a rounding model. I would begin with the domain's required range and acceptable error. A sensor measurement already uncertain by a tenth of a unit has different needs from an exact count of inventory items or a monetary ledger that must preserve specified decimal rounding rules.

The representation should follow those needs. Integers can encode exact counts within a bounded range, fixed-scale integers can represent a declared unit, and decimal arithmetic can express some decimal policies more naturally. Each still has limits, including overflow, scaling and rounding at division. Replacing every floating-point field with a decimal type does not automatically define the business rule. The important decision is what the number means, which operations are permitted and how a caller should interpret the error or rounding introduced by those operations.

Order can change the answer

On ordinary IEEE binary64 arithmetic, adding one to 10^16 can round back to 10^16 because the representable spacing at that magnitude is too large to retain the unit. Subtracting 10^16 afterwards therefore gives zero. Performing the cancellation first and then adding one gives one. The mathematical expression is equivalent over real numbers, but the sequence of rounded operations differs. I would use this small example when reviewing a parallel reduction, because distributing work across lanes or workers often changes the addition order.

The consequence is not that every reordered result is unacceptable. Many applications permit a small numerical difference and benefit from faster or more stable accumulation. Others require reproducibility across runs, machines or execution plans. Those are separate requirements. A deterministic reduction order can improve reproducibility while still producing an inaccurate answer for an ill-conditioned input. A numerically better algorithm can produce a different result from the historical implementation. The acceptance test must specify which property matters instead of treating old output bytes as the only possible definition of correctness.

References: [1] David Goldberg: What Every Computer Scientist Should Know About Floating-Point Arithmetic

Tolerance needs scale and a zero policy

An absolute tolerance bounds error in the result's units. A relative tolerance bounds error in relation to magnitude. Near zero, a purely relative comparison can become unhelpful because the allowed difference shrinks with the reference value. Python's isclose combines relative and absolute criteria according to its documented rule. I would choose both from the domain rather than copy a familiar epsilon. A tolerance appropriate for metres is not automatically appropriate for kilometres, and a tolerance appropriate for a measured quantity may be wrong for an exact identifier.

Suppose an illustrative calculation is allowed an absolute error of 0.001 units near zero and a relative error of one part per million at larger magnitudes. That is a policy with interpretable units and scale, although it still needs validation against the domain. It should not be applied indiscriminately to intermediate values. A sequence of individually tolerated local errors can produce an unacceptable final error. I would state whether the budget applies to the final output, each transformation or a proven propagated bound across the whole computation.

import math

values = [1e16, 1.0, -1e16]
naive = 0.0
for value in values:
    naive += value
print({'naive': naive, 'careful_sum': math.fsum(values)})
assert math.isclose(0.0005, 0.0, rel_tol=1e-6, abs_tol=0.001)
assert not math.isclose(0.002, 0.0, rel_tol=1e-6, abs_tol=0.001)

References: [2] Python: math module

Conditioning and algorithm stability are different

A problem is sensitive when small changes in its inputs can cause large changes in its answer. An algorithm can add avoidable numerical error on top of that inherent sensitivity. I would distinguish these before changing precision. Subtracting two nearly equal measured quantities can produce a small difference whose relative uncertainty is large, even if the subtraction itself is performed accurately. More bits cannot recover information that was absent from the input measurements. The appropriate output may need an uncertainty interval or a warning about sensitivity.

A more stable formulation can nevertheless avoid unnecessary loss. Summation methods that account for rounding, rearrangements that avoid subtracting nearly equal intermediate values and scaling that prevents avoidable overflow can materially improve results. The right technique depends on the computation. I would compare against a higher-precision or analytically known reference on adversarial inputs, while remembering that the reference itself needs a credible error model. Random values in a comfortable range are useful coverage, but they are poor substitutes for deliberately constructed cancellation and extreme-scale cases.

Exceptional values need ordinary interface decisions

NaN and infinity should not be left to propagate without a domain policy. They can be useful representations for invalid operations or unbounded results, but a downstream sort, serialization format or threshold check may not handle them as the caller expects. A comparison involving NaN can fail to behave like an ordinary total ordering. I would decide at the boundary whether non-finite inputs are rejected, preserved with explicit status or transformed according to a documented rule. Silently converting them to zero erases the reason the result became exceptional.

Signed zero can matter in some operations and be irrelevant in others. Similarly, exact equality is perfectly sensible when checking whether a value is exactly an agreed sentinel or whether two deterministic encodings match. The blanket advice never compare floats for equality is too broad. The real question is what relationship the test intends to express. Approximate physical agreement, exact representational identity and ordering for a database key are different relations. Give each a deliberate implementation and test its behaviour for the full admitted value set.

Optimization can spend the error budget

Parallel reductions, fused operations and relaxed-math compiler settings can alter rounding or exceptional-value behaviour. Sometimes that change improves accuracy; sometimes it violates reproducibility or a domain invariant. I would review numeric semantics alongside performance when enabling such transformations. A speed comparison without an error comparison is incomplete if the algorithm's output changes. The budget should specify the acceptable deviation, the population of inputs and the consequences of exceeding it, rather than declare all differences harmless because they occur in low-order bits.

The counterargument is that strict reproducibility can be expensive and unnecessary for noisy data. I agree where the output contract allows it. A scientific exploratory plot and an audited financial statement can reasonably require different policies. The solution is to make the trade explicit, not force one rule on every numerical program. If a faster approximation is acceptable, document its validated range and error bound or empirical error evidence. Outside that range, retain a fallback or reject the input instead of extending confidence beyond the tested domain.

Test numerical meaning, not only sample outputs

A useful numerical test suite includes exact small cases, extreme magnitudes, near-cancellation, non-finite values where admitted and invariants derived from the domain. It can compare multiple formulations or higher-precision references and report absolute and relative errors separately. Metamorphic properties, such as a known scaling relationship, can expose mistakes without requiring a stored answer for every input. Each property needs qualifications: overflow and rounding may make an exact algebraic identity inappropriate. The test model should be more precise than the expression being optimized.

I would record the representation, rounding policy, tolerances and reference method with the implementation. That makes a later compiler change or vectorized path reviewable. Floating-point arithmetic becomes less mysterious when error is treated as a resource with units, bounds and ownership. The question is not whether rounding exists. It is whether the system's computation, representation and validation keep the resulting uncertainty within the promise made to the caller, including the difficult inputs that ordinary examples rarely exercise.

When no defensible tolerance exists, that is a signal to revisit the representation or requirement. An arbitrary epsilon can hide an under-specified contract as effectively as a bare equality check. I would rather expose the missing policy than tune a constant until a collection of familiar fixtures happens to pass.