Exercise 2: Why "Use the Smallest Possible h" Is Wrong — Possible Solution ==================================================================== THE COLLEAGUE'S CLAIM ------------------------------ That accuracy keeps improving indefinitely as h shrinks, so the smallest h a language's floating-point format allows should give the best possible numerical derivative. WHY THIS IS WRONG, PER THIS CHAPTER'S OWN VERIFIED FINDING ------------------------------ This chapter directly verified that the approximation error DOES shrink steadily as h shrinks, but only down to a point - around h=1e-8 to h=1e-10 in the chapter's own x^2 example. Past that point, the error stops shrinking and instead gets dramatically WORSE: at h=1e-13 the error jumped back up to about 0.004 (worse than several much larger h values already tried), and by h=1e-16 the computed result was completely wrong - 0 instead of the true value of 6. WHAT ACTUALLY HAPPENS AS h SHRINKS TOO FAR ------------------------------ When h becomes extremely small, f(x+h) and f(x) become extremely close together in value. Subtracting two nearly-identical floating- point numbers destroys most of their significant digits (this is sometimes called catastrophic cancellation) - the tiny, genuinely meaningful difference between f(x+h) and f(x) gets swamped by floating-point representation error, which is then divided by an equally tiny h, amplifying that error dramatically in the final result. RESULT ------------------------------ The colleague's advice is wrong because numerical differentiation accuracy is NOT a simple "smaller h is always better" relationship - it's a trade-off with a genuine sweet spot. Too large an h gives a poor approximation of the true instantaneous rate of change; too small an h destroys precision through floating-point subtraction error. The best h is somewhere in between, not at either extreme. WHY THIS WORKS AS AN ANSWER ------------------------------ The explanation is grounded directly in this chapter's own verified numbers (the specific error values at h=1e-10 through h=1e-16) rather than a generic statement about floating-point error, and identifies the actual mechanism (subtracting two nearly-identical numbers) causing the breakdown, rather than just asserting "very small h is bad" without explaining why.