Exercise 3: The Three Named Differences Between This Course and LLMs, With Examples — Possible Solution ==================================================================== DIFFERENCE 1: SCALE ------------------------------ Per this chapter, nlp1-9's own GloVe embeddings were trained on billions of words but produced only a static lookup table of word vectors — a large amount of training data feeding a comparatively narrow output (fixed vectors, one per word, that never change once loaded). An LLM trains on comparably massive or larger corpora but learns an entire deep network end to end, capturing far more than static word meaning. The concrete example: this capstone's own `nn.Embedding.from_pretrained(glove_vectors, freeze=True)` line loads a fixed table that cannot represent anything about how a word behaves differently in different contexts — an LLM's own parameters, by contrast, are the whole model, not a lookup table sitting in front of one. DIFFERENCE 2: SELF-SUPERVISED PRETRAINING, TAKEN FURTHER ------------------------------ Per nlp1-5 and nlp1-9, word2vec and GloVe already used a self- supervised training signal — predicting a word from its surrounding context, requiring no hand-labeled data at all. This chapter's own point is that LLMs use the identical underlying idea (predict the next token from context) but scaled up to an entire massive corpus, and that this single objective turns out to be sufficient to learn grammar, facts, sentiment, and style simultaneously. The concrete example: this course had to teach sentiment analysis (nlp1-6) and named entity recognition (nlp1-7) as two separate chapters with two separate training objectives — capabilities a sufficiently large next-token- prediction pretraining run picks up together, as a side effect of one single training signal. DIFFERENCE 3: ONE FLEXIBLE ARCHITECTURE VS. MANY TASK-SPECIFIC PIPELINES ------------------------------ Per Exercise 2's own reasoning, nlp1-6's sentiment classifier and nlp1-7's NER tagger are genuinely different pipelines — different output layers, different objectives, separately trained models — despite sharing the same LSTM mechanism underneath. This chapter's own point is that an LLM, once pretrained, can be steered toward sentiment classification, NER, translation, and more, often through prompting alone, without needing a genuinely new architecture built for each one. The concrete example: this capstone's own model literally cannot perform nlp1-7's NER task, and nlp1-7's own model literally cannot perform this capstone's sentiment classification — two hand-built pipelines each doing exactly one job, contrasted against a single LLM capable of being pointed at both. WHY ALL THREE DIFFERENCES ARE GENUINELY SEPARATE, NOT RESTATEMENTS OF EACH OTHER ------------------------------ Scale describes how much is learned and stored; self-supervised pretraining taken further describes what training signal makes that learning possible without hand-labeled data; one flexible architecture vs. many pipelines describes the practical downstream consequence — how many separate systems are needed to cover a range of tasks. Each addresses a different aspect of what changed, which is why this chapter lists all three rather than treating "bigger" as a single, sufficient explanation. WHY THIS WORKS AS AN ANSWER ------------------------------ It explains each of the three named differences using this chapter's own language, and grounds each one in a specific, concrete example drawn from earlier chapters in this course (GloVe's fixed table, nlp1-6/nlp1-7's separate training objectives, and the two models' mutual inability to perform each other's task) rather than restating the differences in the abstract.