Exercise 1: Why Joint Training Is a Genuinely Different Approach, Not Just a Scale Difference — Possible Solution
====================================================================

HOW nlp1-5's WORD2VEC WAS TRAINED
------------------------------
Per nlp1-5, word2vec is trained from scratch, as its own dedicated
step, on whatever corpus is on hand — a self-supervised prediction
task run in isolation, producing a finished set of embeddings before
any downstream task ever uses them.

HOW nlp1-9's GLOVE WAS TRAINED AND USED
------------------------------
Per nlp1-9, GloVe is pretrained separately, at much larger scale, and
then loaded into a downstream model either frozen (never updated
during that model's own training) or fine-tuned (updated further, but
still starting from a separately-produced starting point). In both
cases, embedding training and downstream-task training are two
distinct phases, even when fine-tuning blurs the boundary somewhat.

HOW AN LLM's OWN TOKEN EMBEDDINGS ARE TRAINED
------------------------------
Per this chapter, an LLM's own embedding table is "trained jointly,
from the very start, alongside every other parameter in the entire
network — there is no separate embedding-training phase at all." The
embedding table isn't produced first and then plugged into a larger
system; it is one of many parameter groups being adjusted simultaneously
by the exact same training process that's also adjusting every
attention and feed-forward parameter in the network.

WHY THIS IS A DIFFERENT APPROACH, NOT JUST A BIGGER VERSION OF nlp1-9's OWN FINE-TUNING
------------------------------
Fine-tuning (nlp1-9) still starts from embeddings that were shaped by a
separate, prior training run with its own separate objective (GloVe's
own co-occurrence-based training) before ever being exposed to the
downstream task. An LLM's own embeddings never have that separate prior
existence — from the very first training step, the exact same
end-to-end objective that shapes the attention layers is also shaping
the embedding table. There is no point at which the embeddings could be
described as "already trained" and then handed off to a separate
process; embedding training and the rest of the model's training are
literally the same process happening at the same time.

WHY THIS MATTERS BEYOND TERMINOLOGY
------------------------------
Because the embeddings are shaped by the exact same signal that shapes
every other part of the network, they can specialize precisely to
whatever representation is most useful for that network's own specific
architecture and task — something a separately pretrained table like
GloVe's, however good in general, cannot guarantee, since it was
optimized for a different, more general objective before ever meeting
this particular model.

WHY THIS WORKS AS AN ANSWER
------------------------------
It lays out, using this chapter's own compare-table, exactly how each
of the three approaches trains its embeddings, and explains precisely
why joint end-to-end training is a structurally different relationship
between embedding and model — not simply GloVe's own fine-tuning taken
further — since the embeddings in an LLM never exist as a separately
finished artifact at any point.