Computers cannot generally store every real number exactly because their numerical representations have finite storage capacity. Instead, they use finite precision formats that retain a limited number of significant binary digits. As a result, many real numbers must be approximated. Arithmetic operations performed with these representations may introduce rounding errors. Such errors can accumulate during long calculations or become amplified by sensitive algorithms. Understanding finite precision arithmetic is essential for writing reliable numerical software and interpreting computational results.
Finite Precision Arithmetic
Finite precision arithmetic is numerical computation using a limited number of digits to represent and manipulate real numbers.
fl(a ∘ b) = (a ∘ b)(1 + δ), |δ| ≤ u
Explanation
Variables
| Symbol | Meaning |
|---|---|
fl | The floating-point result of a computation. |
a, b | Input numerical values. |
∘ | A basic arithmetic operation, when the model applies. |
δ | Relative rounding error in the simplified model. |
u | A bound related to unit roundoff. |
Example or application
In many programming environments, the decimal number 0.1 cannot be represented exactly in binary floating-point arithmetic, so its stored value is a nearby representable number.
Definition
Finite precision arithmetic is arithmetic performed using numerical representations that contain a limited number of digits or bits, causing some real numbers and arithmetic results to be approximated.
Purpose
To explain why computer arithmetic differs from ideal real-number arithmetic and how limited numerical representation affects computations.
Importance
Finite precision is a major source of numerical error. Understanding it helps programmers and scientists avoid unstable calculations, choose suitable numerical formats and interpret small computational discrepancies.
Learning Objectives
Explain why computers use finite precision.
Understand the difference between real arithmetic and floating-point arithmetic.
Identify rounding error.
Recognize that some decimal fractions have repeating binary expansions.
Understand why arithmetic results may differ slightly from mathematical expectations.
Key Concepts
Finite precision
Floating-point arithmetic
Binary representation
Rounding error
Unit roundoff
Representable numbers
Arithmetic approximation
Working Principle
A computer represents numerical values using a finite number of bits. Arithmetic operations produce exact mathematical results conceptually, but the results must often be rounded to the nearest representable value in the chosen format.
Process
Represent the input numbers in the chosen numerical format.
Perform the arithmetic operation.
Determine whether the exact result is representable.
Round or otherwise approximate the result if necessary.
Store the finite precision result.
Analyze the effect of rounding on later computations.
Applications
Scientific programming
Computer simulations
Financial computation
Engineering software
Numerical linear algebra
Machine learning
Worked Example
Problem
Understand why a decimal number such as 0.1 may not be stored exactly in binary floating-point.
Steps
The decimal fraction 0.1 equals 1/10.
Its binary expansion is repeating rather than terminating.
A finite binary floating-point format stores only a limited number of bits.
The stored value is therefore a nearby representable binary number.
Arithmetic using that stored value may produce a tiny difference from ideal real-number arithmetic.
Answer
The stored representation of 0.1 is generally an approximation in binary floating-point formats.
Common Misconceptions
Computers do not represent every real number exactly.
Finite precision error is not necessarily caused by a programming mistake.
Floating-point numbers are not simply ordinary decimal numbers stored with unlimited accuracy.
Small rounding errors can become significant in sensitive calculations.
Practice Questions
| Question | Answer |
|---|---|
What is finite precision arithmetic? | Arithmetic performed using numbers represented with a limited number of digits or bits. |
Why may 0.1 not be represented exactly in binary? | Because its binary expansion repeats and cannot be stored completely using a finite number of bits. |
What is rounding error? | The difference introduced when a mathematical value is replaced by a nearby representable value. |
Key Takeaway
Computer arithmetic uses finite numerical representations, so rounding and representation errors are unavoidable considerations in numerical computation.
Prerequisites
SCI-MAT-014-011
SCI-MAT-014-012
SCI-MAT-014-017
Binary number systems
Reference
Goldberg, D. What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Higham, N. J. Accuracy and Stability of Numerical Algorithms.
Related Topics
SCI-MAT-014-017
SCI-MAT-014-018
SCI-MAT-014-019
SCI-MAT-014-020
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.