Error Propagation & Conditioning

Numerical Methods & Floating-Point Computation

Chapter 6 · Error Propagation & Conditioning

Chapter 5 established that stability — how much a particular algorithm amplifies rounding error — is a real, distinct, fixable property. This chapter covers the other half of the picture: conditioning — how sensitive the underlying mathematical problem itself is to small changes in its input, completely independent of which algorithm is used to solve it. The distinction matters enormously in practice: a stability problem can be fixed by choosing a better algorithm; a conditioning problem often can't be.

The Condition Number, Defined

For a function y = f(x), the relative condition number is cond = |x · f'(x) / f(x)| — the ratio between the relative change in the output and the relative change in the input, for a small perturbation. A condition number near 1 means input errors pass through roughly unchanged. A large condition number means the problem itself amplifies whatever error already exists in the input, regardless of how carefully the evaluation is carried out.

Verified directly — a well-conditioned and an ill-conditioned function, side by side

For f(x) = x² at x=2: the formula predicts cond = |2 · 4 / 4| = 2. Verified by actually perturbing x by a relative 10⁻⁸ and measuring the resulting relative change in f(x): the measured amplification is 2.00000001 — matching the formula almost exactly. For f(x) = 1/(x−1) at x=1.001 (close to the function's singularity at x=1): the formula predicts cond = |1.001/0.001| = 1001. The same perturbation experiment measures an amplification of ≈1000.99 — again matching closely. The second function is genuinely, measurably 500 times more sensitive to the exact same size of input error.

Reframing Cancellation: It Was Conditioning All Along

Chapters 4 and 5 diagnosed cancellation as an algorithm problem — a bad choice of arithmetic steps. The condition-number framework reveals something sharper: the operation a − b itself has its own condition number, (|a| + |b|) / |a − b|, which explodes whenever a and b are close.

Verified directly — subtraction as an ill-conditioned operation

For a = 1,000,000.1, b = 1,000,000.0 (so a − b = 0.1): the subtraction's own condition number is (|a|+|b|)/|a−b| = 20,000,001. Perturbing only a by a relative 10⁻¹² and re-computing a − b exactly (via 50-digit precision arithmetic, so no algorithm-level rounding error is involved at all) produces a relative output change of 10,000,001 times larger than the input perturbation — matching the theoretical a/(a−b) bound for a single-variable perturbation almost exactly.

Resolving an apparent contradiction with Chapters 4-5

This isn't a different phenomenon from cancellation — it's the same phenomenon, now with a precise name and number attached. But it clarifies something important: subtraction of near-equal numbers is ill-conditioned as an operation, yet Chapter 4's stable quadratic-formula reformulation still fixed it. The resolution is that a larger computation can often be restructured to avoid ever performing that specific ill-conditioned subtraction at all — routing the same overall calculation through a different, well-conditioned sequence of operations instead. Algorithm choice doesn't make a specific ill-conditioned operation less sensitive; it can choose not to use that operation when a better-conditioned path to the same answer exists.

When There's No Way Around It: A Genuinely Ill-Conditioned Problem

Sometimes there is no alternative algorithm to switch to, because the sensitivity is baked into the problem as stated. Consider solving the linear system:

x +      y = 2
x + 1.0001y = 2.0001
                  (true solution: x = 1, y = 1)
    

Geometrically, these are two nearly-parallel lines — their determinant (1 × 1.0001 − 1 × 1 = 0.0001) is tiny, meaning the lines intersect at a very shallow angle. A tiny shift in either line moves their intersection point a lot.

Verified directly — exact, rounding-free arithmetic still can't save this

Solving this system with Cramer's rule using 50-digit exact Decimal arithmetic — deliberately eliminating every possible source of algorithm-level rounding error — gives the correct answer, x=1, y=1. Now perturb just one coefficient, 1.0001 → 1.0001 + 10⁻¹⁰ (a relative change of only 10⁻¹⁰, smaller than a typical floating-point rounding error) and solve again, still with the same exact 50-digit arithmetic: x shifts to 0.999998999899... — a relative change of about 10⁻⁶. That's an amplification of roughly 10,000×, and it happened with zero algorithm-level rounding error anywhere in the computation. The entire distortion came from the problem's own sensitivity to its input.

The real-world implication

This is what makes conditioning different from stability in practice: no amount of clever algorithm design fixes an ill-conditioned problem, because the sensitivity doesn't come from how the arithmetic is organized — it comes from the problem itself. If the coefficients 1 and 1.0001 in this system came from real-world measurements with any uncertainty at all, no algorithm, however perfectly implemented, could recover a trustworthy answer — the honest response is to recognize the problem is ill-conditioned and either obtain more precise inputs or accept a wide uncertainty band on the answer, not to search for a better solver.

Stability vs. Conditioning, Side by Side

	Stability (Ch.5)	Conditioning (this chapter)
What it measures	How much rounding error a specific algorithm introduces and amplifies	How much a small input change moves the true, exact answer
Property of	The algorithm / method	The mathematical problem itself
Can it be fixed by switching algorithms?	Yes — Chapters 4-5's whole point	No — a different algorithm solves the same ill-conditioned problem just as badly
Worked example this chapter	(recap) naive variance formula	The near-singular 2-equation linear system

The combined picture

A trustworthy numerical result needs both: a well-conditioned problem (or an honest acknowledgment that it isn't one) and a stable algorithm solving it. A stable algorithm applied to an ill-conditioned problem still gives an unreliable answer — and an unstable algorithm can make even a perfectly well-conditioned problem look unreliable. Chapters 7 and 8 apply exactly this combined lens to root-finding and Gaussian elimination.

Where This Connects

This chapter's finding	What it sets up
The condition number formula `\|x f'(x)/f(x)\|`	Chapter 7's Newton's method, whose own convergence behavior depends directly on the derivative near the root
Near-singular systems amplify input error regardless of algorithm	Chapter 8's ill-conditioned matrices, where the same near-parallel-lines geometry reappears at larger scale
Stability (fixable) vs. conditioning (often not)	The honest diagnostic framework Chapter 10's capstone audit applies to real code

Hands-On Exercises

Exercise 1

Using this chapter's own condition number formula |x f'(x)/f(x)|, compute the condition number of f(x) = ln(x) at x=1.001 (where f'(x) = 1/x), and explain in your own words what a very large value would tell you about evaluating the logarithm near that point.

📄 View solution

Exercise 2

A colleague says "Chapter 4 proved that using a better algorithm fixes catastrophic cancellation, but this chapter says subtraction of near-equal numbers is fundamentally ill-conditioned and can't be fixed — those two chapters contradict each other." Using this chapter's own resolution, explain why they don't actually contradict.

📄 View solution

Exercise 3

Using this chapter's own verified linear-system example, explain why "just switch to a different, more sophisticated equation-solving algorithm" would not actually fix the problem, and describe what a genuinely honest response to this situation would look like instead.

📄 View solution

Chapter 6 Quick Reference

Condition number: cond = |x f'(x)/f(x)| — the ratio of relative output change to relative input change; a property of the problem, not the algorithm
Verified: f(x)=x² at x=2 has cond≈2 (well-conditioned); f(x)=1/(x-1) near its singularity has cond≈1001 (ill-conditioned) — measured amplification matched both predictions closely
Subtraction's own condition number, (|a|+|b|)/|a-b|, explains Chapters 4-5's cancellation as ill-conditioning of that specific operation — not a contradiction, since a larger algorithm can often avoid routing through it
Verified: a near-singular 2-equation linear system amplified a 10⁻¹⁰ relative input perturbation into a ≈10⁻⁶ relative output change (≈10,000×) — using exact, rounding-free 50-digit arithmetic, proving the sensitivity came from the problem, not any algorithm
Stability (Ch.5) is fixable by choosing a better algorithm; conditioning (this chapter) generally is not — both are needed for a trustworthy result
Next chapter: Root-finding methods — bisection and Newton's method, where conditioning near the root directly determines convergence behavior