Exercise 3: Is Gradient Descent "Broken" Because It Got Stuck? — Possible Solution
====================================================================

THE COLLEAGUE'S CLAIM
------------------------------
That gradient descent getting stuck in a worse local minimum proves
the algorithm itself is broken and shouldn't be trusted.

WHY THIS MISDIAGNOSES THE ACTUAL SOURCE OF THE PROBLEM
------------------------------
This chapter drew a specific, verified distinction between two
categories of function: convex functions, where gradient descent is
GUARANTEED to reach the global minimum from any starting point (per
Chapter 6's own worked example, verified across every learning rate
that converged at all), and non-convex functions, where multiple
local minima can genuinely exist and the algorithm's outcome depends
on where it starts.

The "getting stuck" behavior only happened on the non-convex
example (the quartic function with two local minima) - it did NOT
happen on Chapter 6's convex example, where every converging
learning rate reached the exact same true minimum regardless of
starting point. This shows the algorithm is doing exactly what it's
mathematically supposed to do in both cases: following the local
downhill direction faithfully. The "problem" isn't that gradient
descent computed something wrong - it's that the FUNCTION being
optimized had more than one basin to fall into.

WHAT WOULD BE A MORE ACCURATE CONCLUSION
------------------------------
Gradient descent is not broken - it is a LOCAL optimization method,
and it behaves exactly as a local method should: it reliably finds
the bottom of whichever basin it happens to start in. Whether that
basin is the global minimum or not depends entirely on the shape of
the function being optimized (specifically, whether it's convex),
not on any flaw in the algorithm's own logic. The correct response
to the "stuck in a local minimum" problem isn't to distrust gradient
descent - it's to use techniques that address the multiple-minima
issue directly (multiple restarts from different starting points,
more sophisticated variants of gradient descent, or other strategies
this course doesn't go into), while still using gradient descent
itself as the core "go downhill" mechanism.

RESULT
------------------------------
The colleague's conclusion conflates a property of the FUNCTION
(non-convexity, having multiple minima) with a flaw in the
ALGORITHM (gradient descent). The algorithm performed correctly in
both this chapter's convex and non-convex examples; only the
function's own shape changed the outcome.

WHY THIS WORKS AS AN ANSWER
------------------------------
The response draws directly on this chapter's own convex-vs-non-
convex distinction (rather than a generic defense of gradient
descent) and explicitly separates what the algorithm is
mathematically guaranteed to do from what the function's own shape
determines, correcting the specific logical error in the
colleague's reasoning rather than simply disagreeing with the
conclusion.