Calculus & Optimization's own Chapter 1 ran into a real problem and deliberately left it half-explained: shrinking a numerical-differentiation step size h improves accuracy for a while, then suddenly makes it much worse. Linear Algebra Fundamentals solved real systems of equations and computed real matrix inverses, always on numbers that behaved politely. Algorithms & Complexity measured how many operations an algorithm performs — but never asked whether each individual operation gives back a trustworthy answer. This course asks that question directly: when a computer stores and manipulates real numbers, exactly how much can you trust the result, and when does that trust quietly break down?

The Problem, Stated Precisely

Almost every programmer eventually runs this in some language and is surprised by it:

This chapter deliberately doesn't yet explain why this happens at the bit level — that's Chapter 2's job. What matters here is that this is not a rare edge case. It is the default, everyday behavior of floating-point arithmetic, and it has real, verifiable consequences for ordinary code.

A Real Consequence: A Loop That Doesn't Do What It Looks Like It Does

Consider a loop written by someone who reasonably expects that adding 0.1 ten times gives exactly 1.0, or that repeatedly adding 0.1 will eventually land exactly on 1.0 and stop:

Scale the same idea up and the error compounds rather than staying microscopic: adding the value 0.10 one million times — the kind of thing a naive running-total accumulator in a billing or accounting system might do — produces a result off from the mathematically exact answer by roughly 1.33 × 10⁻⁶. Still small in absolute terms, but no longer negligible once real money or a strict equality check is on the other end of that number, and the error grows with every additional operation rather than staying fixed.

What This Course Actually Covers

What This Course Won't Cover

Numerical computing as a full field is enormous, and much of it is out of scope for what a working programmer actually needs day to day:

• Symbolic / exact computer algebra — systems like a computer algebra system that manipulate exact fractions or symbolic expressions sidestep floating-point error entirely by design; this course is about the arithmetic every mainstream language actually performs by default, not about avoiding it
• GPU-specific and parallel floating-point quirks — reduced-precision formats, non-associative parallel summation order, and hardware-specific fused multiply-add behavior are real and important for high-performance/ML infrastructure work, but are a specialized extension of this course's own fundamentals, not covered here
• Arbitrary-precision and interval arithmetic libraries — genuinely useful tools that trade speed for guaranteed precision; this course focuses on understanding and working with standard double-precision floats, the default nearly every language reaches for first

Where This Course Is Headed

Hands-On Exercises

Chapter 1 showed that 0.1 + 0.2 != 0.3, verified down to the exact numbers involved, but deliberately stopped short of explaining why. This chapter opens the box: the standard almost every language uses to store a real number — IEEE 754 — and the specific reason a number as ordinary as 0.1 can never be stored exactly in it.

The Three-Part Layout: Sign, Exponent, Mantissa

A standard double-precision float (64 bits — double in C/Java, the only numeric float type in JavaScript, Python's default float) splits its bits into three fields:

The stored value is reconstructed as (-1)^sign × (1 + mantissa/2⁵²) × 2^(exponent−1023). The "1 +" is the implicit leading bit — every normal number is assumed to start with a binary 1.xxxxx, so that one bit doesn't need to be stored at all, silently giving 53 bits of precision from only 52 stored mantissa bits.

Why 0.1 Specifically Can Never Be Exact

Every stored float is, structurally, a binary fraction times a power of two — nothing else is representable. Expanding 0.1 as a binary fraction by repeated doubling (the standard technique, exactly analogous to long division for decimal expansions):

Not every decimal fraction has this problem — 0.5, 0.25, and 0.125 are all exact powers of two (2⁻¹, 2⁻², 2⁻³) and store perfectly, with an all-zero mantissa. The problem is specifically fractions whose denominator, in lowest terms, isn't a pure power of 2.

Single vs. Double Precision

Python's built-in float, JavaScript's Number, Java/C#'s double, and C's double are all double-precision by default. Single precision (float in C/Java, Float32Array in JavaScript) shows up mainly where memory or speed matters more than precision — graphics, GPUs, and large numeric arrays.

Subnormal Numbers: Gradual Underflow

The exponent field's all-zero value (0) is reserved as a special signal: it means "drop the implicit leading 1 bit" and switch to subnormal representation, allowing numbers smaller than the smallest normal float, at the cost of gradually losing precision as they shrink.

The Special Values: Infinity, NaN, and Signed Zero

Two more reserved exponent patterns give IEEE 754 its special values, alongside a genuine oddity: zero has two distinct bit patterns.

Where This Connects

Hands-On Exercises

Chapter 2 explained why a value like 0.1 gets rounded when it's stored. This chapter puts an exact number on how much that rounding can possibly be — the single most important quantity in this whole course for reasoning precisely about floating-point trustworthiness, rather than just gesturing at "some small error."

Absolute Error vs. Relative Error

Absolute error is simply |approximation − true value| — the raw size of the gap. Relative error divides that gap by the true value's own size: |approximation − true value| / |true value|. The same absolute error can mean something completely different depending on the scale of the number involved.

Machine Epsilon, Derived — Not Quoted

Machine epsilon is the smallest positive number ε such that 1.0 + ε is a different, representable floating-point value from 1.0 itself. Rather than simply stating its value, it can be found directly with a genuinely simple experiment: start with ε = 1.0 and keep halving it until adding it to 1.0 stops making a difference.

Machine epsilon is not "the smallest number a computer can represent" — Chapter 2's subnormal numbers go vastly smaller than this. It specifically measures precision relative to a value near 1.0 — how fine-grained the gaps between adjacent representable floats are at that scale. Because of the floating-point format's own sliding exponent, this same relative gap size holds at any magnitude — which is exactly why Chapter 2's "absolute vs. relative error" distinction matters so much here.

The Four IEEE Rounding Modes

Whenever an arithmetic result doesn't fit exactly into the available mantissa bits, it has to be rounded to the nearest representable value — and IEEE 754 defines specific, precise rules for how that rounding happens. Four modes are the ones exposed by virtually every language and standard library:

(Honest note: the IEEE 754 standard technically also defines a fifth rounding attribute, round-ties-away-from-zero, but it's rarely exposed as a default in mainstream hardware or languages the way the four above are — round-to-nearest-ties-to-even is overwhelmingly the one that matters day to day, which is why the rest of this section focuses on it.)

Why "Ties to Even" Specifically — A Verified Demonstration

Round-half-up (the rounding most people learn in school — 0.5 always rounds to 1) seems more intuitive than round-half-even. But it has a real, measurable flaw: applied repeatedly to many values that each happen to land exactly on a .5 boundary, it introduces a systematic upward bias, because every single tie breaks the same direction.

Where This Connects

Hands-On Exercises

Chapter 3 established machine epsilon — the smallest gap between representable values near 1.0. This chapter shows the single most common way that gap turns into a real, visible bug: subtracting two floating-point numbers that are very close to each other. The operation itself is completely correct by IEEE 754's own rules — it's what subtraction does to your remaining precision that causes the damage.

The Core Mechanism

Every floating-point number carries roughly 15-17 significant decimal digits of precision (Chapter 2's 52-bit mantissa). When two numbers that agree in their first several digits are subtracted, those matching leading digits cancel out exactly — but the trailing digits that made each number only approximately correct in the first place don't cancel; they're what's left over. The result can end up built almost entirely out of rounding error, even though the subtraction itself was performed with full precision.

The Classic Case: An Unstable Quadratic Formula

The quadratic formula, x = (−b ± √(b² − 4ac)) / (2a), is taught as a single, universal recipe — but implemented naively in floating point, it has a real, well-known failure mode whenever b is much larger than a and c: one of the two ± branches subtracts two nearly-equal numbers.

The standard fix — sometimes called the "Citardauq" formula (read the standard quadratic formula's name backward) — restructures the computation so the two large, same-signed quantities are always added, never subtracted, and the small root is obtained by division instead:

Resolved: Calculus & Optimization's Own Unexplained Finding

Calculus & Optimization Chapter 1 verified that numerical differentiation, (f(x+h) − f(x)) / h, gets more accurate as h shrinks — until it suddenly gets catastrophically worse. That course stated the observation honestly but left the mechanism for this course to explain. It is exactly the cancellation pattern above.

Where This Connects

Hands-On Exercises

Chapter 4 fixed one specific formula — the quadratic formula's unstable branch — by restructuring a single subtraction. This chapter generalizes that idea: numerical stability is a property of an algorithm, not of the underlying mathematical problem. Two formulas that are perfectly, provably equal in exact arithmetic can behave completely differently once real floating-point rounding gets involved — and recognizing which formula you're looking at is a skill in its own right.

Two Equivalent Formulas for the Same Quantity: Variance

Statistical variance has two textbook-equivalent formulas. The two-pass formula computes the mean first, then averages the squared distance of each point from that mean. The one-pass ("naive") formula avoids a second pass over the data by expanding the algebra: Var(X) = E[X²] − (E[X])².

Algebraically, these are exactly the same quantity — every statistics textbook derives one from the other with a few lines of expansion. In floating point, they are not remotely the same.

A Genuinely Dramatic, Fully Verified Failure

Take 9 real numbers, each equal to 20,000,000 plus a small random offset between 0 and 1 (a realistic shape for, say, sensor readings with a large fixed baseline and small genuine variation):

The two-pass formula never runs into this problem, because it subtracts the mean from each data point first — before any squaring or summing happens. Every value it works with afterward is already small and close to zero (each of the 9 offsets, centered), so there are no huge intermediate sums for rounding error to hide inside.

It's Not Always Catastrophic — But It's Never Free

The size of the failure depends on how large the data's offset is relative to its own spread. Using the same 9-point idea but with a smaller offset (around 1,000,000 instead of 20,000,000), the naive formula's relative error was verified at roughly 0.25% — a real, measurable error, but not one that produces an outright impossible result. Push the offset larger still (around 10⁸), and the naive formula's result collapses to exactly 0.0 — complete, 100% information loss, the same complete-collapse pattern Chapter 4 verified for (1+x)-1 at small enough x.

Numerical Stability, Defined

An algorithm is numerically stable if small rounding errors introduced at each individual step stay small in the final result, relative to the size of what's being computed. An algorithm is numerically unstable if those small per-step errors can be amplified into a large, disproportionate error in the output — exactly what the naive variance formula does whenever the data's own scale is much larger than its spread. Crucially, stability is a property of how the computation is organized, not of the mathematical quantity being computed — both formulas compute "variance"; only one of them computes it reliably.

Where This Connects

Hands-On Exercises

Chapter 5 established that stability — how much a particular algorithm amplifies rounding error — is a real, distinct, fixable property. This chapter covers the other half of the picture: conditioning — how sensitive the underlying mathematical problem itself is to small changes in its input, completely independent of which algorithm is used to solve it. The distinction matters enormously in practice: a stability problem can be fixed by choosing a better algorithm; a conditioning problem often can't be.

The Condition Number, Defined

For a function y = f(x), the relative condition number is cond = |x · f'(x) / f(x)| — the ratio between the relative change in the output and the relative change in the input, for a small perturbation. A condition number near 1 means input errors pass through roughly unchanged. A large condition number means the problem itself amplifies whatever error already exists in the input, regardless of how carefully the evaluation is carried out.

Reframing Cancellation: It Was Conditioning All Along

Chapters 4 and 5 diagnosed cancellation as an algorithm problem — a bad choice of arithmetic steps. The condition-number framework reveals something sharper: the operation a − b itself has its own condition number, (|a| + |b|) / |a − b|, which explodes whenever a and b are close.

When There's No Way Around It: A Genuinely Ill-Conditioned Problem

Sometimes there is no alternative algorithm to switch to, because the sensitivity is baked into the problem as stated. Consider solving the linear system:

Geometrically, these are two nearly-parallel lines — their determinant (1 × 1.0001 − 1 × 1 = 0.0001) is tiny, meaning the lines intersect at a very shallow angle. A tiny shift in either line moves their intersection point a lot.

Stability vs. Conditioning, Side by Side

Where This Connects

Hands-On Exercises

Finding an x where f(x) = 0 is one of the most common tasks in applied computing — and it's where this course's earlier chapters stop being abstract cautionary tales and start directly shaping how a real algorithm is built. This chapter covers three approaches with three genuinely different reliability profiles: bisection (always works, but slowly), Newton's method (usually fast, but can fail outright — reusing Calculus & Optimization's own derivative rules directly), and fixed-point iteration, the general framework both of the others turn out to be special cases of.

Bisection: Guaranteed, Slow, and Hard to Break

If f(a) and f(b) have opposite signs, the Intermediate Value Theorem guarantees a root exists somewhere between them. Bisection exploits this directly: check the midpoint's sign, keep whichever half still brackets the root, and repeat. Every single iteration is guaranteed to halve the interval containing the root — no more, no less.

Newton's Method: Reusing Calculus & Optimization's Own Derivative Rules

Newton's method uses the derivative to take a far more informed step: approximate f near the current guess with its tangent line (exactly the derivative Calculus & Optimization Chapter 3 defined), and jump to where that tangent line crosses zero: x_{n+1} = x_n − f(x_n)/f'(x_n).

Two Genuine Ways Newton's Method Fails

Newton's speed comes at a real cost: unlike bisection, it has no built-in guarantee. Two distinct, verified failure modes:

Failure 1 — A Permanent Cycle

Failure 2 — Degraded Convergence at a Double Root

Newton's method divides by f'(x) at every step — which is exactly the kind of division Chapter 6 flagged as dangerous when the denominator is close to zero. At a double root (where both f(x)=0 and f'(x)=0 at the same point), that's precisely what happens as the iteration converges.

Fixed-Point Iteration: The General Framework Underneath Both

A fixed-point iteration repeatedly applies x_{n+1} = g(x), hoping the sequence settles at a point where x = g(x). The convergence rule is precise: the iteration converges near a fixed point if |g'(x)| < 1 there, and diverges if |g'(x)| > 1 — a direct, exact echo of Chapter 6's condition-number logic, now applied to a repeated process instead of a single evaluation.

Where This Connects

Hands-On Exercises

Linear Algebra Fundamentals covered Gaussian elimination as an exact, always-correct algebraic procedure. In floating point, it is not always safe to run exactly as written — the order in which rows are eliminated can make the difference between an accurate answer and a badly wrong one, on the very same system of equations.

Plain Gaussian Elimination Can Fail — Even Without a Genuinely Ill-Conditioned System

Solve this ordinary-looking 2×2 system:

Standard Gaussian elimination uses the first row's leading entry as the pivot to eliminate x from the second row. Here, that pivot is 1 × 10⁻¹⁶ — tiny, but not literally zero, so nothing raises an error. The elimination step computes a multiplier m = 1 / 10⁻¹⁶ = 10¹⁶ and uses it to combine the rows.

The Fix: Partial Pivoting

Partial pivoting is a simple, mechanical rule: before eliminating a column, swap rows so that the row with the largest available magnitude in that column becomes the pivot row. It doesn't change the mathematical system being solved — just the order the rows are processed in.

Ill-Conditioned Matrices, Revisited With Real Numbers

Partial pivoting fixes an unstable algorithm (Chapter 5's territory) — but Chapter 6 already showed some systems are ill-conditioned problems, which no algorithm can fix. The matrix condition number, cond(A) = ‖A‖ · ‖A⁻¹‖, puts a precise number on this, computed directly using Linear Algebra Fundamentals' own determinant and inverse formulas.

Two Genuinely Different Diagnoses, Side by Side

Where This Connects

Hands-On Exercises

Calculus & Optimization Chapters 4 and 9 introduced numerical differentiation and integration as basic techniques and this course's own Chapter 4 explained why the simplest version breaks down. This chapter goes further: three genuinely better techniques for differentiation, and one genuinely better strategy for integration — each directly informed by what Chapters 1-8 already established about cancellation, stability, and conditioning.

Central Differences: A Free Upgrade Over Forward Differences

The forward-difference formula, (f(x+h)-f(x))/h, uses one point ahead of x. The central-difference formula uses one point on each side: (f(x+h) − f(x−h)) / (2h). Both approximate the same derivative, but their error behaves very differently as h shrinks — forward error shrinks proportionally to h itself, while central error shrinks proportionally to h².

Complex-Step Differentiation: Sidestepping Cancellation Entirely

Both formulas above eventually fail because they subtract two nearly-equal real numbers. Complex-step differentiation is a genuinely different trick: evaluate the function at a small imaginary step, f(x + ih), and take the imaginary part: f'(x) ≈ Im(f(x+ih)) / h. For a function that's analytic (most ordinary functions — polynomials, sin, exp, etc. all qualify), this formula has essentially no cancellation error, because it never subtracts two real, nearly-equal quantities at all — the real and imaginary parts of a complex number are stored and manipulated independently.

Richardson Extrapolation: Cancel the Leading Error Term

Central differences have error that behaves predictably: D(h) = f'(x) + C·h² + O(h⁴) for some constant C. Richardson extrapolation exploits this directly — compute the same central-difference estimate at two step sizes, h and h/2, and combine them to cancel out the h² term algebraically: R = (4·D(h/2) − D(h)) / 3. What's left over is O(h⁴) — a dramatically better estimate, built entirely from two ordinary central-difference calculations already in hand.

Adaptive Quadrature: Spending Effort Where It's Needed

Calculus & Optimization Chapter 9 covered fixed-grid numerical integration — evaluating the function at evenly-spaced points across the whole interval, regardless of how the function actually behaves. Adaptive quadrature instead estimates the local error on each subinterval and only subdivides further where that error is too large — concentrating function evaluations where the function is actually changing quickly, and spending almost none where it's nearly flat.

Where This Connects

Hands-On Exercises

One continuous audit: a small, ordinary-looking industrial sensor-monitoring codebase, module by module, each one carrying a real floating-point bug of exactly the kind this course has spent nine chapters teaching how to recognize. Every diagnosis below reuses this course's own already-verified numbers directly — nothing here is re-derived from scratch, because the whole point of a real audit is applying knowledge you already trust, not re-proving it each time.

Step 1 — The Calibration Validity Check