Exercise 2: Why the Local Maximum Is an Unstable Starting Point — Possible Solution ==================================================================== THE GRADIENT AT THE CRITICAL POINT ITSELF ------------------------------ By this chapter's own definition, x=0.559 is a critical point specifically because f'(0.559) is essentially 0 (verified directly: computing f'(x) at this exact location gives a value extremely close to zero, consistent with it being one of the three critical points this chapter already identified). WHAT THIS MEANS FOR GRADIENT DESCENT STARTING EXACTLY THERE ------------------------------ Gradient descent's own update rule is x_new = x_old - alpha*f'(x). If f'(x) is exactly 0, the update does literally nothing - x_new = x_old - alpha*0 = x_old. Mathematically, a gradient descent search starting EXACTLY at a critical point would never move at all, regardless of which kind of critical point it is (minimum, maximum, or otherwise) - the algorithm has no information at that single point telling it which direction to go. WHY THIS IS SPECIFICALLY UNSTABLE HERE (NOT JUST STATIONARY) ------------------------------ This chapter's own second-derivative test showed x=0.559 has f''(x) < 0 - a local MAXIMUM, not a minimum. In practice, no real starting point is ever exactly, infinitely precisely at the true critical value - floating-point representation, or any tiny real-world perturbation, places the actual starting point a minuscule distance to one side or the other. Once even slightly off this maximum, the gradient is no longer exactly zero, and gradient descent moves DOWNHILL away from the maximum - but which way it moves (toward the shallow minimum or the deep global one) depends entirely on which side of the maximum that tiny perturbation landed on. This chapter's own verified finding (starting at x=0 vs x=1, straddling this exact maximum) showed those two nearby starting points landing in completely different, far-apart basins. RESULT ------------------------------ Starting gradient descent exactly at x=0.559 is unreliable specifically because it sits on a local maximum, where the gradient vanishes but any infinitesimal deviation sends the search rolling toward one of two very different outcomes - unlike starting near a minimum, where nearby starting points reliably converge back to the same result. WHY THIS WORKS AS AN ANSWER ------------------------------ The explanation connects the zero-gradient fact directly to what the update rule literally computes there, and explains the instability specifically in terms of this chapter's own already- verified finding about x=0 and x=1 landing in different basins, rather than a generic statement about local maxima being "bad."