✦ScienexySCIENCE KNOWLEDGE LIBRARY
concept · SCI-MAT-014-019

Machine Precision

Machine precision describes the resolution of floating-point numbers near 1 and is closely related to the smallest relative spacing between representable values.

u = 2⁻ᵖ

Explanation

A computer's floating-point format can represent only a finite set of numbers. Machine precision describes the scale of the spacing between nearby representable values, particularly around 1. In numerical analysis, unit roundoff is commonly used to bound the relative error introduced by correctly rounded arithmetic under standard assumptions. For a common binary floating-point format with p significant binary digits, the spacing near 1 and the unit roundoff are closely related but are not always identical under every convention. Machine precision is important because it helps explain why two mathematically different values may be stored as the same floating-point number and why very small changes can disappear during computation.

Variables

SymbolMeaning

u

Unit roundoff for a common binary rounding-to-nearest model.

p

Number of significant binary digits in the significand, including the implicit leading bit where applicable.

Example or application

In the common IEEE 754 binary64 format, the significand precision is 53 binary digits. Its unit roundoff under round-to-nearest is 2⁻⁵³.

Definition

Machine precision is a measure of the resolution of a computer's floating-point number system, commonly described through the spacing of representable values near 1 or through a related unit-roundoff quantity.

Purpose

To explain the numerical resolution and rounding limitations of computer arithmetic.

Importance

Machine precision helps determine whether a difference between two values can be represented, whether a numerical tolerance is realistic and how rounding errors may influence algorithms.

Learning Objectives

  • Define machine precision.

  • Understand the relationship between precision and representable-number spacing.

  • Distinguish machine epsilon from unit roundoff.

  • Recognize the influence of machine precision on calculations.

  • Understand why extremely small numerical changes may be lost.

Key Concepts

  • Machine precision

  • Machine epsilon

  • Unit roundoff

  • Floating-point spacing

  • Binary precision

  • Rounding

  • Representable numbers

Working Principle

Machine precision is determined by the structure of the floating-point format. The number of significant bits determines the spacing between nearby representable values and provides a scale for rounding-error analysis.

Process

  • Identify the floating-point format.

  • Determine its significand precision.

  • Establish the spacing between representable values near 1.

  • Determine the unit-roundoff convention being used.

  • Compare the required numerical tolerance with the available precision.

Applications

  • Scientific computing

  • Numerical error analysis

  • Floating-point algorithm design

  • Computer simulations

  • Numerical testing

  • Programming language arithmetic

Worked Example

Problem

Find the unit roundoff for a binary floating-point system with p = 4 significant binary digits under the stated model.

Steps

  • Use u = 2⁻ᵖ.

  • Substitute p = 4.

  • Calculate u = 2⁻⁴.

  • Therefore, u = 1/16 = 0.0625.

Answer

The unit roundoff is 0.0625 under the stated binary rounding model.

Common Misconceptions

  • Machine epsilon and unit roundoff are related but may differ by a factor depending on their definitions.

  • Machine precision is not the same as the number of decimal digits printed on the screen.

  • Increasing the displayed decimal places does not increase the underlying floating-point precision.

  • The spacing between representable numbers changes with their magnitude.

Practice Questions

QuestionAnswer

What does machine precision describe?

The resolution of the floating-point representation and the spacing of representable numerical values.

What is the unit roundoff for p = 5 in the model u = 2⁻ᵖ?

u = 2⁻⁵ = 1/32 = 0.03125.

Does floating-point spacing remain constant at every magnitude?

No. The spacing generally changes with the magnitude of the represented values.

Key Takeaway

Machine precision describes the finite resolution of computer arithmetic and provides a foundation for understanding rounding error.

Prerequisites

  • SCI-MAT-014-016

  • SCI-MAT-014-017

  • SCI-MAT-014-018

  • Powers of two

Reference

  • Higham, N. J. Accuracy and Stability of Numerical Algorithms.

  • IEEE Standard for Floating-Point Arithmetic, IEEE 754.

Related Topics

  • SCI-MAT-014-016

  • SCI-MAT-014-018

  • SCI-MAT-014-020

  • SCI-MAT-014-026

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.

Suggest a correction or source · More in Numerical Analysis