Exercise 1: Why This Pipeline Is Fine-Tuning at Full Network Scale, Per nlp1-9's Own Prediction — Possible Solution
====================================================================

WHAT nlp1-9 SAID ABOUT FROZEN VS. FINE-TUNED
------------------------------
Per this chapter's own finding-box, nlp1-9 established the frozen-vs-
fine-tuned distinction specifically for a lookup table of pretrained
word embeddings — freezing means using the pretrained vectors as-is;
fine-tuning means continuing to update them, starting from that
pretrained state, using a smaller, task-specific dataset. nlp1-9
explicitly stated this shape mirrors "an entire pretrained network"
being either used as-is or adapted further, with "only the scale"
differing between the two cases.

WHAT THIS CHAPTER'S OWN PIPELINE ACTUALLY DOES TO THE MODEL
------------------------------
Per this chapter, SFT "continues training on these prompt/response
pairs, using the identical next-token-prediction objective from
llm1-7" — starting from the base model's own already-pretrained
weights (llm1-7) and continuing to update them, but now using a much
smaller, carefully curated dataset instead of the massive raw
pretraining corpus. This is structurally identical to nlp1-9's own
definition of fine-tuning: begin from a broadly-trained starting point,
then continue adapting it using a smaller, more targeted dataset.

WHY "ENTIRE PRETRAINED NETWORK" IS THE ACCURATE DESCRIPTION HERE
------------------------------
nlp1-9's own fine-tuning updated only the embedding table — a single,
comparatively small component of a larger system. This chapter's own
SFT step updates the weights of the entire Transformer network built up
through llm1-5 (every attention matrix, every feed-forward layer, not
merely an embedding lookup table). The object being fine-tuned is now
the model in its entirety, rather than one component sitting in front
of it — precisely the "far larger scale" nlp1-9 predicted, applied to
the identical underlying logic.

WHY THE REWARD MODEL AND RLHF STEPS EXTEND THIS SAME LOGIC FURTHER
------------------------------
The reward-modeling and RLHF steps continue this same broadly-trained-
then-further-adapted pattern one level further: RLHF starts from the
SFT model's own already fine-tuned weights and adapts them again, this
time using a reward signal derived from human preference rankings
rather than direct imitation examples. Each step is another round of
"start from what's already been learned, then adapt further using a
smaller, more specific signal" — the same nlp1-9 logic, applied
repeatedly at increasing specificity.

WHY THIS WORKS AS AN ANSWER
------------------------------
It restates nlp1-9's own frozen-vs-fine-tuned framework and its own
explicit prediction about scale, then shows precisely how this
chapter's own SFT step matches that framework's logic while updating
the entire pretrained network rather than an embedding table, and
extends the explanation to the reward-modeling and RLHF steps as
further rounds of the identical underlying adaptation pattern.