Exercise 3: Gradient Descent and Backpropagation Are the Same Idea — Possible Solution
====================================================================

WHAT THIS CHAPTER'S OWN CONNECTIONS TABLE SAYS
------------------------------
The table lists gradient descent (Chapter 6) as "the literal
training algorithm behind linear regression, logistic regression,
and every deep neural network," and separately lists the chain
rule/backpropagation (Chapter 8) as "how a neural network actually
learns - computing how every internal weight should change."

WHY THESE AREN'T ACTUALLY TWO SEPARATE TOPICS
------------------------------
Gradient descent is the OUTER procedure: repeatedly nudge every
parameter in the direction that reduces the cost function, using
the gradient (the vector of partial derivatives) to know which
direction that is. This works identically whether the "model" being
optimized is a simple one-parameter formula or a neural network
with millions of parameters - the loop itself doesn't change.

What changes for a neural network specifically is HOW that gradient
gets computed in the first place. A neural network is a deeply
nested, composed function (layers feeding into layers feeding into
a final output), so computing "how does the cost change with
respect to a weight buried three layers deep" requires the chain
rule, applied repeatedly through that nested structure -
backpropagation is simply the name for that repeated chain-rule
application, done efficiently.

RESULT
------------------------------
Gradient descent is the general-purpose optimization procedure;
backpropagation is the specific technique for computing the
gradient THAT gradient descent needs, when the function being
optimized happens to be a deeply nested one like a neural network.
They aren't competing or alternative ideas - backpropagation feeds
its output directly into gradient descent's own input requirement.
Chapter 6 builds the general procedure; Chapter 8 builds the
specific gradient-computation technique a neural network needs to
actually use that procedure.

WHY THIS WORKS AS AN ANSWER
------------------------------
The explanation identifies the specific relationship between the
two chapters (one is the general optimization loop, the other is
how to efficiently compute the gradient that loop needs for a
particular kind of function) rather than simply restating that both
are "used in machine learning," which wouldn't actually explain why
they're the same underlying idea rather than two coincidentally
related ones.