Capstone — Diagnosing and Fixing Numerical Bugs in Real Code
Numerical Methods & Floating-Point Computation
Chapter 10 · Capstone — Diagnosing and Fixing Numerical Bugs in Real Code
One continuous audit: a small, ordinary-looking industrial sensor-monitoring codebase, module by module, each one carrying a real floating-point bug of exactly the kind this course has spent nine chapters teaching how to recognize. Every diagnosis below reuses this course's own already-verified numbers directly — nothing here is re-derived from scratch, because the whole point of a real audit is applying knowledge you already trust, not re-proving it each time.
| Step | Module audited | Bug type | Chapter(s) used |
|---|---|---|---|
| 1 | Calibration validity check | Exact-equality comparison | Ch.1-2 |
| 2 | Daily totals aggregator | Rounding bias | Ch.3 |
| 3 | Drift-rate calculator | Catastrophic cancellation | Ch.4, Ch.9 |
| 4 | Sensor variance monitor | Unstable variance formula | Ch.5 |
| 5 | Dual-sensor calibration system | Ill-conditioned matrix | Ch.6, Ch.8 |
| 6 | Set-point solver | Newton's method cycling | Ch.7 |
| 7 | Two-sensor blend solver | Unpivoted elimination | Ch.8 |
| 8 | Trajectory integrator | Wasteful fixed-step integration | Ch.9 |
Step 1 — The Calibration Validity Check
Two sensors are supposed to sum to a known reference value of 0.3. The check almost never passes, even for correctly calibrated sensors.
0.1 + 0.2 evaluates to 0.30000000000000004, which differs from the literal 0.3 (stored as 0.29999999999999998...) by ≈5.55×10⁻¹⁷ — exactly the gap Chapter 1 verified. The exact-equality check was never going to reliably pass, regardless of how correctly the sensors were actually calibrated.
Step 2 — The Daily Totals Aggregator
Thousands of individual readings are rounded to the nearest whole unit before being summed into a daily report, using each language's default "round half up" behavior.
.5 boundary with round-half-up produces a systematic +500 bias versus the true total, while round-half-to-even produces zero bias. The aggregator has exactly this shape: any sensor reading that happens to land on an exact half-unit tie gets rounded the same direction every time, so the daily total drifts upward in proportion to how many exact ties occur — a real, compounding, one-directional error, not random noise that averages out.
Fix: switch the rounding mode to round-half-to-even (IEEE 754's own default, and Python's built-in round()) rather than a manually-implemented round-half-up.
Step 3 — The Drift-Rate Calculator
A well-meaning engineer picked an extremely small h, reasoning "smaller step, more accurate derivative."
f(x)=x² at x=3: at h=10⁻¹³, the relative error in the derivative reaches ≈6.8×10⁻⁴, and by h=10⁻¹⁶ the approximation collapses to 0 entirely, for a true derivative of 6. The engineer's instinct — smaller h is always better — is precisely the mistake this course spent Chapter 4 resolving: shrinking h drives temperature_fn(t+h) and temperature_fn(t) together, triggering catastrophic cancellation in the numerator.
Fix: switch to central differencing at a moderate step size (Chapter 9 verified central differences beat forward differences by nearly five orders of magnitude at h=10⁻⁴), or — since the sensor's own temperature function is a simple, analytic calculation internally — use complex-step differentiation, verified exact to full double precision at any step size down to 10⁻¹⁰⁰, eliminating the tuning problem entirely.
Step 4 — The Sensor Variance Monitor
A pressure sensor with a large fixed baseline reading (around 20,000,000 units) plus small genuine fluctuation triggers repeated false hardware-fault alerts.
−0.0625 — a mathematically impossible negative variance — on real data with a large baseline offset, against a true variance of ≈0.0673. The sensor's own large baseline is triggering the identical bug: E[X²] and (E[X])² are both enormous, and their difference (the true, tiny variance) is swamped by the rounding error accumulated while computing those two huge sums.
Fix: switch to the two-pass formula (center the data around its own mean first, then square) — Chapter 5 verified this matches a high-precision reference to ≈7×10⁻¹⁷ relative error on the identical data.
Step 5 — The Dual-Sensor Calibration System
Two redundant pressure sensors are calibrated together by solving a small linear system relating their readings. The computed calibration constants vary wildly between runs, even on nearly identical input data.
[[1,1],[1,1.0001]], with a verified matrix condition number of ≈40,004. The two sensors are giving almost redundant information (nearly parallel equations), and Chapter 6 proved directly — using exact, rounding-free 50-digit arithmetic — that no algorithm can fix this: a 10⁻¹⁰ relative input perturbation still produced a ≈10⁻⁶ relative output change, a genuine property of the sensor pair's own geometry, not of any solver.
Fix: this is a hardware/sensor-placement issue, not a software one — per Chapter 6's own honest conclusion, the correct response is to compute and report the condition number directly, flag the sensor pair as too redundant to calibrate reliably, and recommend physically repositioning one sensor rather than continuing to search for a better solver.
Step 6 — The Set-Point Solver
For one particular valve's calibration curve, f(x)=x³−2x+2, starting from a "reasonable" default guess of x₀=0, the solver never returns.
0 → 1 → 0 → 1 → ... cycle under Newton's method — not slow convergence, but no convergence at all, from a starting point that looked entirely reasonable.
Fix: add an iteration cap and a bracketing fallback — if Newton's method hasn't converged within a set number of steps, fall back to bisection (Chapter 7's own guaranteed-but-slower alternative) using a bracket confirmed by a sign change, rather than looping indefinitely.
Step 7 — The Two-Sensor Blend Solver
A separate calibration routine blends two sensor readings by solving a small linear system via plain Gaussian elimination, no row swapping. For one particular pair of sensitivity coefficients, one of the two blended outputs comes back wildly wrong.
10⁻¹⁶-pivot example. Verified there: one output variable came back with a relative error of ≈1.22 (more than 100% wrong) while the other stayed accurate to ≈10⁻¹⁶ — the huge elimination multiplier (10¹⁶) amplified rounding error straight through Chapter 4's own cancellation mechanism during back-substitution.
Fix: add partial pivoting to the elimination routine. Chapter 8 verified this alone drops the error from ≈1.22 down to ≈10⁻¹⁶ — the exact same system, correctly solved, just by changing the row processing order.
Step 8 — The Trajectory Integrator
A separate module numerically integrates sensor-derived acceleration data into a trajectory estimate using a fixed, uniformly-spaced grid — chosen once, years ago, and never revisited. Most flight segments are smooth, but a small number involve a sharp maneuver the fixed grid consistently under-resolves.
501 function evaluations to reach an error of ≈3.16×10⁻⁹, while adaptive quadrature reaches comparable accuracy (≈3.53×10⁻⁹) using only 265 evaluations — roughly half — because most of the fixed grid's points are wasted on the smooth stretches while the sharp maneuver is exactly where more resolution is actually needed.
Fix: replace the fixed-step integrator with adaptive quadrature. Sharp maneuvers automatically get more evaluation points where they're needed; smooth stretches automatically get fewer — better accuracy on the segments that matter, at lower total computational cost.
Audit Summary
| Step | Root cause | Fixable by better code? |
|---|---|---|
| 1 | Exact equality on a value that was never guaranteed to be exact | Yes — tolerance comparison |
| 2 | Directionally-biased rounding compounding over many operations | Yes — round-to-even |
| 3 | Cancellation from an over-aggressively small step size | Yes — central or complex-step differencing |
| 4 | An algebraically-unstable one-pass formula | Yes — two-pass formula |
| 5 | An inherently ill-conditioned sensor pair | No — hardware issue, honestly reported |
| 6 | An unguarded iterative method with no fallback | Yes — iteration cap + bisection fallback |
| 7 | An unstable elimination order | Yes — partial pivoting |
| 8 | A fixed grid wasting effort on well-behaved regions | Yes — adaptive quadrature |
What This Course Doesn't Cover
As stated honestly back in Chapter 1: symbolic/exact computer algebra, GPU-specific and parallel floating-point quirks (reduced precision, non-associative summation order, fused multiply-add), and arbitrary-precision/interval arithmetic libraries were all named as deliberately out of scope, and stayed out of scope through all ten chapters. Every bug in this capstone was diagnosed and fixed using only standard double-precision arithmetic — the default nearly every language reaches for first, and exactly the territory this course committed to from the start.
Where This Course Connects
Calculus & Optimization's own Chapter 1 forward-referenced the exact cancellation mechanism resolved in this course's Chapter 4, and its own numerical differentiation/integration (Chapters 4 and 9) were the introductory versions of what this course's Chapter 9 covered in real depth. Linear Algebra Fundamentals' determinant and inverse formulas were used directly to compute a real matrix condition number in Chapter 8. Algorithms & Complexity's own iterative-algorithm framing underlies both Chapter 7's root-finding and Chapter 9's adaptive quadrature. Within Technical Support, perfdiag1's own performance-diagnosis discipline and appdiag1's own root-cause reasoning are the same "measure, don't guess" instinct this capstone applied to numerical bugs specifically.
Hands-On Exercises
A colleague proposes fixing Step 5's ill-conditioned dual-sensor system by "just using higher-precision floating point (128-bit) instead of standard doubles." Using this chapter's own Step 5 diagnosis and Chapter 6's original reasoning, explain whether this would actually fix the problem.
📄 View solutionUsing this chapter's own Audit Summary table, group the eight bugs into two categories: those caused by an algorithm choice (fixable by using a different, better algorithm for the exact same problem) and those caused by the problem's own inherent sensitivity (not fixable by algorithm choice alone). Justify each grouping using this course's own Chapter 5/6 distinction.
📄 View solutionStep 6's fix adds an iteration cap and a bisection fallback to the set-point solver, rather than simply switching entirely from Newton's method to bisection. Using this chapter's own Step 6 diagnosis and Chapter 7's original comparison of the two methods, explain why keeping Newton's method as the primary approach (with bisection only as a fallback) is a better design than replacing it outright.
📄 View solutionChapter 10 Quick Reference
- Full worked audit: eight modules, eight bugs, each diagnosed by directly reusing an already-verified finding from Chapters 1 through 9 rather than re-deriving anything from scratch
- Seven of the eight bugs were genuine algorithm/code issues, fixable with a better formula, a tolerance check, pivoting, or an adaptive method
- One bug (Step 5) was not a code bug at all — an inherently ill-conditioned sensor pair, correctly diagnosed by computing and honestly reporting a condition number rather than chasing a nonexistent software fix
- Every diagnosis in this capstone traces back to one of three root mechanisms taught across the course: cancellation (Ch.4), instability (Ch.5), or ill-conditioning (Ch.6) — everything else in the course builds on recognizing which of these three is actually at play
- Course complete — Numerical Methods & Floating-Point Computation, 10 chapters, from
0.1+0.2≠0.3to a fully audited, fixed, real codebase