Exercise 2: Why Complex-Step Differentiation Has No Cancellation Error — Possible Solution
====================================================================

WHAT OPERATION IS MISSING
------------------------------
The forward and central difference formulas both compute a
SUBTRACTION between two real numbers that are close in value:
f(x+h) - f(x) or f(x+h) - f(x-h). That subtraction is exactly the
operation Chapter 4 identified as dangerous - the leading digits of
the two nearly-equal operands cancel out, leaving a result built
mostly from whatever rounding error each operand already carried.

The complex-step formula, Im(f(x+ih))/h, involves no such
subtraction at all. It evaluates the function only ONCE, at the
single point x+ih, and then simply reads off the imaginary part of
that one result - there is no second function value being subtracted
from it anywhere in the formula.

WHY THE DIVISION BY h ISN'T A PROBLEM HERE
------------------------------
Dividing by a small h only becomes dangerous when it amplifies error
that already exists in the numerator - which is exactly what
happened in Chapter 4's numerical-derivative example, where the
numerator (f(x+h)-f(x)) had already lost precision to cancellation
before the division even happened. In complex-step differentiation,
the imaginary part of f(x+ih) is not the result of a real-number
subtraction - it's a genuinely separate quantity, computed by the
function's own internal complex arithmetic, that carries essentially
the same floating-point precision regardless of how small h is. Since
there's no cancellation-damaged numerator to amplify, dividing by a
tiny h doesn't reveal any hidden error the way it did in this
course's earlier examples (Chapter 1's resolution of the original
numerical-differentiation breakdown, and Chapter 4's own explanation
of it).

WHY THIS EXPLAINS THE VERIFIED "EXACT AT h=10^-100" RESULT
------------------------------
Because the imaginary-part computation never subtracts two real,
nearly-equal quantities, there is no precision to lose as h shrinks -
the only limiting factor becomes how small h can get before h*h
itself underflows to zero in floating point (an entirely different,
much more extreme limit than the cancellation threshold that broke
the real-valued formulas at ordinary small values like 10^-16). This
is exactly why this chapter's own experiment found the same fully
accurate answer at every tested h, all the way down to 10^-100 - far
beyond where either real-valued finite-difference formula could
survive.

WHY THIS WORKS AS AN ANSWER
------------------------------
The explanation identifies the specific missing operation (a
real-number subtraction between two close values) rather than vaguely
crediting "complex numbers are magic," and explains why the
remaining division by h is safe in this formula specifically because
its numerator was never damaged by cancellation in the first place.