IEEE 754 Floating-Point Representation
Numerical Methods & Floating-Point Computation
Chapter 2 · IEEE 754 Floating-Point Representation
Chapter 1 showed that 0.1 + 0.2 != 0.3, verified down to the exact numbers involved, but deliberately stopped short of explaining why. This chapter opens the box: the standard almost every language uses to store a real number — IEEE 754 — and the specific reason a number as ordinary as 0.1 can never be stored exactly in it.
The Three-Part Layout: Sign, Exponent, Mantissa
A standard double-precision float (64 bits — double in C/Java, the only numeric float type in JavaScript, Python's default float) splits its bits into three fields:
1 bit
11 bits (bias 1023)
52 bits
The stored value is reconstructed as (-1)^sign × (1 + mantissa/2⁵²) × 2^(exponent−1023). The "1 +" is the implicit leading bit — every normal number is assumed to start with a binary 1.xxxxx, so that one bit doesn't need to be stored at all, silently giving 53 bits of precision from only 52 stored mantissa bits.
0.1's 64 bits, read directly: sign = 0, raw exponent = 1019 (unbiased: 1019 − 1023 = −4), mantissa = 0x999999999999a. Plugging back into the formula: (1 + 0x999999999999a / 2⁵²) × 2⁻⁴ reconstructs to exactly the same value Python stores for 0.1 — confirming the formula, not just asserting it.
Why 0.1 Specifically Can Never Be Exact
Every stored float is, structurally, a binary fraction times a power of two — nothing else is representable. Expanding 0.1 as a binary fraction by repeated doubling (the standard technique, exactly analogous to long division for decimal expansions):
0.1's binary expansion is infinitely repeating — the pattern 0011 never terminates, exactly the way 1/3 never terminates in decimal (0.333...). A computer only has 52 mantissa bits to work with, so it must cut that infinite pattern off and round — which is precisely the ...999999999999a tail seen in 0.1's decomposition above. There is no larger number of bits that fixes this; any finite binary format has the same problem, because the issue is the repeating pattern itself, not a shortage of bits. This is exactly the same phenomenon as decimal being unable to store 1/3 exactly — floating point just hits it far more often, because so many ordinary decimal fractions (0.1, 0.2, 0.3, 0.7...) turn out to be repeating in binary even though they terminate cleanly in decimal.
Not every decimal fraction has this problem — 0.5, 0.25, and 0.125 are all exact powers of two (2⁻¹, 2⁻², 2⁻³) and store perfectly, with an all-zero mantissa. The problem is specifically fractions whose denominator, in lowest terms, isn't a pure power of 2.
Single vs. Double Precision
| Format | Total bits | Sign | Exponent | Mantissa | Decimal digits of precision |
|---|---|---|---|---|---|
Single (float32) | 32 | 1 | 8 (bias 127) | 23 | ~7 |
Double (float64) | 64 | 1 | 11 (bias 1023) | 52 | ~15-17 |
0.1 decomposed as a 32-bit float: sign = 0, raw exponent = 123 (unbiased −4 — identical exponent to the double, since 0.1's magnitude doesn't change), mantissa = 0x4ccccd. The unbiased exponent matches the double exactly; only the mantissa is shorter (23 bits instead of 52), which is exactly why single precision is less accurate at representing 0.1, not differently rounded in some unrelated way.
Python's built-in float, JavaScript's Number, Java/C#'s double, and C's double are all double-precision by default. Single precision (float in C/Java, Float32Array in JavaScript) shows up mainly where memory or speed matters more than precision — graphics, GPUs, and large numeric arrays.
Subnormal Numbers: Gradual Underflow
The exponent field's all-zero value (0) is reserved as a special signal: it means "drop the implicit leading 1 bit" and switch to subnormal representation, allowing numbers smaller than the smallest normal float, at the cost of gradually losing precision as they shrink.
| Value | What it is |
|---|---|
| ≈ 2.2250738585072014 × 10⁻³⁰⁸ | Smallest normal positive double (exponent field = 1, smallest non-subnormal) |
| ≈ 4.9406564584124654 × 10⁻³²⁴ | Smallest subnormal positive double — verified exponent field = 0, mantissa = 1 (a single bit) |
0.0 — a discontinuous cliff. Subnormals fill that gap with a smoothly shrinking (if increasingly imprecise) sequence of tiny nonzero values instead, which matters for numerical algorithms — covered later in this course — that rely on results changing continuously rather than snapping to zero.
The Special Values: Infinity, NaN, and Signed Zero
Two more reserved exponent patterns give IEEE 754 its special values, alongside a genuine oddity: zero has two distinct bit patterns.
| Value | Bit pattern signal | Verified behavior |
|---|---|---|
Infinity (±∞) | Exponent all 1s, mantissa all 0 | Represents overflow / division results too large to store — e.g. 1.0 / float('inf') == 0.0, verified directly |
| NaN ("not a number") | Exponent all 1s, mantissa nonzero | Represents an undefined result (e.g. 0/0). Verified directly: nan == nan is False — NaN is defined to compare unequal to everything, including itself |
Signed zero (+0.0 / -0.0) | All bits zero except (for -0.0) the sign bit | Verified directly: 0.0 == -0.0 is True (they compare equal) yet their raw bit patterns genuinely differ (0000...0000 vs 8000...0000) — and division reveals the difference: 1.0 / +0.0 and 1.0 / -0.0 would produce +∞ and -∞ respectively under IEEE division |
1.0 / 0.0 as +Infinity, and JavaScript and Java both follow that directly. Python deliberately overrides this at the language level and raises ZeroDivisionError instead, treating it as a program error rather than a valid floating-point result — verified directly in this environment. The underlying hardware and the IEEE 754 standard agree on what should happen; the language you're using can still choose to intercept it.
nan != nan is True — the only value in IEEE 754 that is never equal to itself — x != x is a real, working way to test whether a floating-point value x is NaN, without needing a dedicated isnan() function at all (though using one, like Python's math.isnan(), is clearer and the recommended approach in real code).
Where This Connects
| This chapter's finding | What it resolves or sets up |
|---|---|
| 0.1's infinite repeating binary expansion | Directly resolves Chapter 1's own unexplained 0.1 + 0.2 != 0.3 finding — now traced to its exact bit-level cause |
| 52 stored mantissa bits, 53 with the implicit leading bit | Sets up Chapter 3's formal definition of machine epsilon — the precise size of the smallest gap between representable numbers |
| Rounding a value to fit the mantissa | Sets up Chapter 4's catastrophic cancellation, which is fundamentally about what happens when that rounding is subtracted out |
Hands-On Exercises
Using this chapter's own reconstruction formula (-1)^sign × (1 + mantissa/2⁵²) × 2^(exponent−1023) and the verified decomposition of 1.0 (sign=0, raw exponent=1023, mantissa=0), show step by step that the formula reconstructs exactly 1.0.
Explain, using this chapter's own binary-expansion argument, why 0.5 stores in a float with zero error, while 0.1 and 0.2 do not. Your answer should reference what makes a fraction's binary expansion terminate versus repeat forever.
A function receives a floating-point value x from an untrusted external source and needs to check whether it's a valid number before using it in a calculation. Using this chapter's own verified NaN property, explain why a naive check like if x == some_error_sentinel would fail to catch a NaN value, and what check would actually work.
Chapter 2 Quick Reference
- A double-precision float = 1 sign bit + 11 exponent bits (bias 1023) + 52 mantissa bits, with an implicit leading
1giving 53 bits of real precision - Verified:
0.1's binary expansion repeats forever (0011pattern), so it must be rounded to fit 52 bits — the exact, bit-level cause of Chapter 1's0.1+0.2 != 0.3 - Single precision (32-bit) uses 8 exponent bits + 23 mantissa bits — same exponent range logic, far less mantissa precision (~7 vs. ~16 decimal digits)
- Subnormals (exponent field
= 0) allow gradual underflow instead of a sudden jump to zero - Special values:
Infinity(exponent all 1s, mantissa 0),NaN(exponent all 1s, mantissa nonzero, verified never equal to itself), and signed zero (+0.0/-0.0, equal by==but distinct bit patterns) - Verified: Python raises
ZeroDivisionErroron1.0/0.0rather than returning IEEE 754's own defined+Infinity— a language choice layered on top of the standard, not the standard itself - Next chapter: Rounding error and machine epsilon — putting an exact number on how much any of this can go wrong