Log-Space

Log-space represents a positive quantity x by its logarithm. Products become sums because log(ab) = log(a) + log(b) for positive a and b. This is useful when many small positive factors would produce a product too small for the chosen floating-point format.

For example, multiplying two values near 10⁻²⁰⁰ in binary64 produces zero. Adding their natural logarithms instead gives approximately −921.034, a finite value that can still be compared with other log products. Compute these logarithms before multiplication. Taking a logarithm after rounding the product to zero cannot restore the lost value.

Not every small-looking number needs this treatment: 10⁻¹¹⁰ is representable as a nonzero binary64 approximation. The difficulty depends on the final result and the sequence of operations, not the number of zeros printed. If a log result is converted back using exp, the ordinary value may underflow again. Many likelihood calculations can compare log results directly and avoid that conversion.

Adding probabilities requires a different identity: log(a + b) is not log(a) + log(b). A log-sum-exp calculation shifts log inputs by their maximum before exponentiating, sums the shifted values, and adds the maximum back after taking a logarithm. This reduces overflow risk and limits avoidable underflow, but sufficiently small relative contributions may still disappear. Zero is often represented by negative infinity in a log-domain convention; negative values need separate sign handling. Log-space changes the representation, not the domain rules or every source of numerical error.

References

• Python math functions

For worked storage examples, see How Computers Store Data.