A computer's floating-point format can represent only a finite set of numbers. Machine precision describes the scale of the spacing between nearby representable values, particularly around 1. In numerical analysis, unit roundoff is commonly used to bound the relative error introduced by correctly rounded arithmetic under standard assumptions. For a common binary floating-point format with p significant binary digits, the spacing near 1 and the unit roundoff are closely related but are not always identical under every convention. Machine precision is important because it helps explain why two mathematically different values may be stored as the same floating-point number and why very small changes can disappear during computation.
Machine Precision
Machine precision describes the resolution of floating-point numbers near 1 and is closely related to the smallest relative spacing between representable values.
u = 2⁻ᵖ
Explanation
Variables
| Symbol | Meaning |
|---|---|
u | Unit roundoff for a common binary rounding-to-nearest model. |
p | Number of significant binary digits in the significand, including the implicit leading bit where applicable. |
Example or application
In the common IEEE 754 binary64 format, the significand precision is 53 binary digits. Its unit roundoff under round-to-nearest is 2⁻⁵³.
Definition
Machine precision is a measure of the resolution of a computer's floating-point number system, commonly described through the spacing of representable values near 1 or through a related unit-roundoff quantity.
Purpose
To explain the numerical resolution and rounding limitations of computer arithmetic.
Importance
Machine precision helps determine whether a difference between two values can be represented, whether a numerical tolerance is realistic and how rounding errors may influence algorithms.
Learning Objectives
Define machine precision.
Understand the relationship between precision and representable-number spacing.
Distinguish machine epsilon from unit roundoff.
Recognize the influence of machine precision on calculations.
Understand why extremely small numerical changes may be lost.
Key Concepts
Machine precision
Machine epsilon
Unit roundoff
Floating-point spacing
Binary precision
Rounding
Representable numbers
Working Principle
Machine precision is determined by the structure of the floating-point format. The number of significant bits determines the spacing between nearby representable values and provides a scale for rounding-error analysis.
Process
Identify the floating-point format.
Determine its significand precision.
Establish the spacing between representable values near 1.
Determine the unit-roundoff convention being used.
Compare the required numerical tolerance with the available precision.
Applications
Scientific computing
Numerical error analysis
Floating-point algorithm design
Computer simulations
Numerical testing
Programming language arithmetic
Worked Example
Problem
Find the unit roundoff for a binary floating-point system with p = 4 significant binary digits under the stated model.
Steps
Use u = 2⁻ᵖ.
Substitute p = 4.
Calculate u = 2⁻⁴.
Therefore, u = 1/16 = 0.0625.
Answer
The unit roundoff is 0.0625 under the stated binary rounding model.
Common Misconceptions
Machine epsilon and unit roundoff are related but may differ by a factor depending on their definitions.
Machine precision is not the same as the number of decimal digits printed on the screen.
Increasing the displayed decimal places does not increase the underlying floating-point precision.
The spacing between representable numbers changes with their magnitude.
Practice Questions
| Question | Answer |
|---|---|
What does machine precision describe? | The resolution of the floating-point representation and the spacing of representable numerical values. |
What is the unit roundoff for p = 5 in the model u = 2⁻ᵖ? | u = 2⁻⁵ = 1/32 = 0.03125. |
Does floating-point spacing remain constant at every magnitude? | No. The spacing generally changes with the magnitude of the represented values. |
Key Takeaway
Machine precision describes the finite resolution of computer arithmetic and provides a foundation for understanding rounding error.
Prerequisites
SCI-MAT-014-016
SCI-MAT-014-017
SCI-MAT-014-018
Powers of two
Reference
Higham, N. J. Accuracy and Stability of Numerical Algorithms.
IEEE Standard for Floating-Point Arithmetic, IEEE 754.
Related Topics
SCI-MAT-014-016
SCI-MAT-014-018
SCI-MAT-014-020
SCI-MAT-014-026
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.