Why Numerical Methods Matter for Programmers
Numerical Methods & Floating-Point Computation
Chapter 1 · Why Numerical Methods Matter for Programmers
Calculus & Optimization's own Chapter 1 ran into a real problem and deliberately left it half-explained: shrinking a numerical-differentiation step size h improves accuracy for a while, then suddenly makes it much worse. Linear Algebra Fundamentals solved real systems of equations and computed real matrix inverses, always on numbers that behaved politely. Algorithms & Complexity measured how many operations an algorithm performs — but never asked whether each individual operation gives back a trustworthy answer. This course asks that question directly: when a computer stores and manipulates real numbers, exactly how much can you trust the result, and when does that trust quietly break down?
The Problem, Stated Precisely
Almost every programmer eventually runs this in some language and is surprised by it:
0.1 + 0.2 evaluates to 0.30000000000000004440892..., while the literal 0.3 stores as 0.29999999999999998889777.... The two differ by about 5.55 × 10⁻¹⁷ — a genuinely tiny gap, but a real, non-zero one, which is exactly why 0.1 + 0.2 == 0.3 evaluates to False in Python, JavaScript, Java, C, C#, Go, Rust, and effectively every other mainstream language. This isn't a bug in any one of them — it's a direct, shared consequence of how every one of them stores a fractional number in binary, covered in full in Chapter 2.
This chapter deliberately doesn't yet explain why this happens at the bit level — that's Chapter 2's job. What matters here is that this is not a rare edge case. It is the default, everyday behavior of floating-point arithmetic, and it has real, verifiable consequences for ordinary code.
A Real Consequence: A Loop That Doesn't Do What It Looks Like It Does
Consider a loop written by someone who reasonably expects that adding 0.1 ten times gives exactly 1.0, or that repeatedly adding 0.1 will eventually land exactly on 1.0 and stop:
0.1 ten times produces 0.9999999999999999, not 1.0 — off by about 1.1 × 10⁻¹⁶. Worse, the while x != 1.0 loop never terminates as written: x creeps past 1.0 without ever landing on it exactly — after 20 iterations it has already overshot to 2.0000000000000004 and will keep climbing forever, since it can never satisfy x == 1.0. A loop that looks obviously correct by inspection runs indefinitely in practice.
Scale the same idea up and the error compounds rather than staying microscopic: adding the value 0.10 one million times — the kind of thing a naive running-total accumulator in a billing or accounting system might do — produces a result off from the mathematically exact answer by roughly 1.33 × 10⁻⁶. Still small in absolute terms, but no longer negligible once real money or a strict equality check is on the other end of that number, and the error grows with every additional operation rather than staying fixed.
What This Course Actually Covers
| Topic | Where it actually shows up |
|---|---|
| IEEE 754 representation (Ch.2) | Why 0.1 can't be stored exactly in binary in the first place — the root cause behind every example in this chapter |
| Rounding error & machine epsilon (Ch.3) | Putting a precise number on "how wrong" a floating-point value can be, instead of just observing that it is |
| Catastrophic cancellation (Ch.4) | Why Calculus & Optimization Chapter 1's own numerical-derivative approximation broke down as its step size h got too small — resolved directly in this course |
| Numerical stability (Ch.5-6) | Why two mathematically identical formulas can give wildly different answers once real floating-point numbers are involved |
| Root-finding & linear algebra pitfalls (Ch.7-8) | Where Calculus & Optimization's derivatives and Linear Algebra Fundamentals' Gaussian elimination can silently misbehave on real hardware |
What This Course Won't Cover
Numerical computing as a full field is enormous, and much of it is out of scope for what a working programmer actually needs day to day:
- Symbolic / exact computer algebra — systems like a computer algebra system that manipulate exact fractions or symbolic expressions sidestep floating-point error entirely by design; this course is about the arithmetic every mainstream language actually performs by default, not about avoiding it
- GPU-specific and parallel floating-point quirks — reduced-precision formats, non-associative parallel summation order, and hardware-specific fused multiply-add behavior are real and important for high-performance/ML infrastructure work, but are a specialized extension of this course's own fundamentals, not covered here
- Arbitrary-precision and interval arithmetic libraries — genuinely useful tools that trade speed for guaranteed precision; this course focuses on understanding and working with standard double-precision floats, the default nearly every language reaches for first
Where This Course Is Headed
| Chapter | Topic |
|---|---|
| 2 | IEEE 754 Floating-Point Representation |
| 3 | Rounding Error & Machine Epsilon |
| 4 | Catastrophic Cancellation & Loss of Significance |
| 5 | Numerical Stability: When the Algorithm Is the Problem |
| 6 | Error Propagation & Conditioning |
| 7 | Root-Finding Methods |
| 8 | Numerical Linear Algebra Pitfalls |
| 9 | Numerical Differentiation & Integration, Revisited |
| 10 | Capstone — Diagnosing and Fixing Numerical Bugs in Real Code |
Hands-On Exercises
Using this chapter's own verified numbers, explain precisely why 0.1 + 0.2 == 0.3 evaluates to False. Your answer should name the actual gap between the two values, not just say "floating-point is imprecise."
A colleague writes a loop that repeatedly adds 0.1 to a running total and stops with while x != 1.0:. Using this chapter's own verified finding, explain exactly what goes wrong, and propose a one-line fix that would make the loop terminate reliably.
This chapter says the "add 0.1 a million times" error (about 1.33 × 10⁻⁶) grows as more additions happen, rather than staying fixed the way the ten-addition error did. Explain, in your own words and without yet knowing Chapter 3's formal error-propagation rules, why doing more additions with the same small rounding error per step would plausibly make the total error larger rather than smaller.
Chapter 1 Quick Reference
0.1 + 0.2 == 0.3isFalsein effectively every mainstream language — verified: the two sides differ by about5.55 × 10⁻¹⁷- This is a shared, structural consequence of binary floating-point storage, not a bug in any one language — explained fully in Chapter 2
- Verified directly: a
while x != 1.0loop built on repeated0.1additions never terminates, sincexovershoots1.0without ever landing on it exactly - Error from repeated floating-point addition compounds rather than staying fixed — verified: summing
0.1a million times is off by about1.33 × 10⁻⁶ - Course scope: representation, rounding, cancellation, stability, and error propagation for standard double-precision floats — not symbolic/exact computer algebra, GPU-specific quirks, or arbitrary-precision libraries
- Next chapter: IEEE 754 floating-point representation — where the
0.1 + 0.2gap actually comes from, bit by bit