Exercise 1: Why Negative Variance Proves a Bug, Not Just Inaccuracy — Possible Solution ==================================================================== THE MATHEMATICAL FACT THAT MAKES THIS DIAGNOSTIC ------------------------------ Variance is defined as an average of squared deviations from the mean: Var(X) = average of (x - mean)^2 across all data points. Every single term being averaged is a square, and a square of a real number can never be negative. An average of non-negative numbers can never itself be negative. This isn't a property of any particular dataset - it's true for every possible dataset, by the definition of variance itself, before any computation is even attempted. WHY THIS DISTINGUISHES "BUGGY" FROM "SOMEWHAT INACCURATE" ------------------------------ An approximate variance calculation that returns, say, 0.06 when the true value is 0.0673 is still inside the space of mathematically valid variance values - it's simply not precise. That's ordinary, expected floating-point imprecision, the kind Chapters 1-4 already established happens in nearly every computation. A NEGATIVE result, by contrast, isn't merely imprecise - it's a value that no correctly functioning variance computation, run on any real dataset, could ever mathematically produce. Getting a result outside the entire range of valid outputs is qualitatively different from getting an approximately-correct value inside that range. WHAT THIS TELLS US ABOUT THE NAIVE FORMULA SPECIFICALLY ------------------------------ Because negative variance is mathematically impossible in principle, observing it directly proves that the computation's own rounding error became larger than the true quantity being computed - the accumulated error in E[X^2] - (E[X])^2 exceeded the size of the true variance itself, and swamped it entirely. This chapter's own verified example showed exactly this: naive_var = -0.0625 against a true value of about 0.0673 - the rounding error was actually larger in magnitude than the correct answer. WHY THIS WORKS AS AN ANSWER ------------------------------ The explanation grounds the argument in the actual mathematical definition of variance (an average of squares, which forces non-negativity), distinguishes "outside the valid range entirely" from "inside the valid range but imprecise," and connects the conclusion back to this chapter's own verified -0.0625 result as concrete evidence rather than a hypothetical.