09-03-2026, 12:18 AM
Summary
David Goldberg’s classic 1991 paper explains why floating-point arithmetic is fundamentally approximate and why programmers cannot safely treat floating-point numbers like exact real numbers. Computers have only a finite number of bits, so many ordinary numbers—most famously decimal $0.1$—cannot be represented exactly in binary. Consequently, arithmetic operations usually require rounding. Goldberg introduces important ways of measuring this error, including relative error, machine epsilon, and ULPs (units in the last place). He also explains catastrophic cancellation, where subtracting two nearly equal quantities can destroy most meaningful digits. Algebraically equivalent formulas can therefore behave very differently numerically; rewriting expressions can dramatically improve accuracy.
A major part of the paper discusses the IEEE 754 floating-point standard, which was designed to make numerical computation more predictable across computer systems. IEEE arithmetic specifies rounding behavior for basic operations and introduces special values such as $+\infty$, $-\infty$, NaN (Not a Number), $+0$, $-0$, and denormalized/subnormal numbers. These are not merely technical curiosities: infinities allow computations to continue after overflow, NaNs propagate invalid results, signed zero preserves useful mathematical information, and subnormal numbers provide gradual underflow, avoiding an abrupt jump from very small numbers directly to zero. The standard also defines exceptions including overflow, underflow, division by zero, invalid operations, and inexact results.
Goldberg’s broader message is that reliable numerical computing depends not only on hardware precision but also on algorithms, compilers, programming languages, and system design. Compiler optimizations that are mathematically valid over real numbers may be unsafe for floating-point arithmetic because associativity and other familiar algebraic identities do not always hold after rounding. Even IEEE-compliant computers are not guaranteed to produce identical results in every situation because intermediate precision, decimal conversions, mathematical libraries, and compiler behavior can differ. Programmers therefore need to understand numerical stability rather than assuming that simply using more precision will eliminate floating-point problems.
Key takeaways
Overall: Goldberg's central lesson is that floating-point arithmetic is not defective arithmetic—it is a carefully designed approximation system. Problems arise mainly when programmers assume it behaves exactly like mathematics over the real numbers. The paper remains one of the foundational introductions to numerical computing and IEEE floating-point arithmetic.
ARTICLE
David Goldberg’s classic 1991 paper explains why floating-point arithmetic is fundamentally approximate and why programmers cannot safely treat floating-point numbers like exact real numbers. Computers have only a finite number of bits, so many ordinary numbers—most famously decimal $0.1$—cannot be represented exactly in binary. Consequently, arithmetic operations usually require rounding. Goldberg introduces important ways of measuring this error, including relative error, machine epsilon, and ULPs (units in the last place). He also explains catastrophic cancellation, where subtracting two nearly equal quantities can destroy most meaningful digits. Algebraically equivalent formulas can therefore behave very differently numerically; rewriting expressions can dramatically improve accuracy.
A major part of the paper discusses the IEEE 754 floating-point standard, which was designed to make numerical computation more predictable across computer systems. IEEE arithmetic specifies rounding behavior for basic operations and introduces special values such as $+\infty$, $-\infty$, NaN (Not a Number), $+0$, $-0$, and denormalized/subnormal numbers. These are not merely technical curiosities: infinities allow computations to continue after overflow, NaNs propagate invalid results, signed zero preserves useful mathematical information, and subnormal numbers provide gradual underflow, avoiding an abrupt jump from very small numbers directly to zero. The standard also defines exceptions including overflow, underflow, division by zero, invalid operations, and inexact results.
Goldberg’s broader message is that reliable numerical computing depends not only on hardware precision but also on algorithms, compilers, programming languages, and system design. Compiler optimizations that are mathematically valid over real numbers may be unsafe for floating-point arithmetic because associativity and other familiar algebraic identities do not always hold after rounding. Even IEEE-compliant computers are not guaranteed to produce identical results in every situation because intermediate precision, decimal conversions, mathematical libraries, and compiler behavior can differ. Programmers therefore need to understand numerical stability rather than assuming that simply using more precision will eliminate floating-point problems.
Key takeaways
- Floating-point numbers are approximations, not exact real numbers.
- Numbers such as $0.1$ generally cannot be represented exactly in binary.
- Rounding errors are normally tiny, but algorithms can amplify them dramatically.
- Subtracting nearly equal calculated quantities can cause catastrophic cancellation.
- Numerically stable reformulations of formulas are often more important than simply increasing precision.
- IEEE 754 provides standardized rounding plus special values such as NaN, $\pm\infty$, $\pm0$, and subnormal numbers.
- Floating-point arithmetic does not always obey ordinary algebraic rules; for example, $(a+b)+c$ need not equal $a+(b+c)$.
- Reliable scientific software requires cooperation between algorithms, hardware, compilers, and programming languages.
Overall: Goldberg's central lesson is that floating-point arithmetic is not defective arithmetic—it is a carefully designed approximation system. Problems arise mainly when programmers assume it behaves exactly like mathematics over the real numbers. The paper remains one of the foundational introductions to numerical computing and IEEE floating-point arithmetic.
ARTICLE
┌────────────────────────────────┐
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘