Exercise 1: The Architectural Difference Between Sequence Labeling and nlp1-6's Sequence Classification — Possible Solution ==================================================================== WHAT nlp1-6's OWN MODEL DID WITH ITS HIDDEN STATES ------------------------------ Per nlp1-6, the LSTM produced a sequence of hidden states as it processed the sentence step by step, but the classifier head only ever looked at the FINAL one: embedded = self.embedding(x) _, (hidden, _) = self.lstm(embedded) return self.output(hidden[-1]) # only the last hidden state Every hidden state except the very last was computed and then simply discarded. The final hidden state was treated as a single summary of the entire sentence, and one sigmoid-activated prediction (positive or negative sentiment) was produced from that one summary. WHAT THIS CHAPTER'S OWN MODEL DOES INSTEAD ------------------------------ This chapter's own code keeps every hidden state: outputs, _ = self.lstm(embedded) # every hidden state, not just the last return self.output(outputs) # one prediction per token The underlying `nn.LSTM` call is producing the exact same sequence of per-step hidden states nlp1-6's model already computed internally — the only change is which part of that output is used. `outputs` here holds every time step's own hidden state, stacked together, rather than the LSTM's own `hidden` return value holding just the final one. WHY FEEDING EACH HIDDEN STATE THROUGH THE SAME CLASSIFIER PRODUCES PER-TOKEN OUTPUT ------------------------------ `self.output` is still the same kind of small linear classifier head nlp1-6 used — but instead of applying it once, to one summary vector, it is applied independently to every individual hidden state in the sequence. Since there is one hidden state per token (per nn1-8's own step-by-step mechanism, reused unchanged from nlp1-6), applying the same classifier to each one produces exactly one prediction per token — sequence labeling — instead of exactly one prediction for the whole sequence — sequence classification. WHY THIS COUNTS AS "SMALLER THAN IT SOUNDS" ------------------------------ Nothing about the LSTM itself changed — it was always computing a full sequence of hidden states internally; nlp1-6 simply chose to use only the last one. The entire difference between sequence classification and sequence labeling comes down to which of the LSTM's own already- computed outputs get passed to the classifier head, and how many times the classifier head is applied. WHY THIS WORKS AS AN ANSWER ------------------------------ It identifies the exact single line that changed between the two models (using `outputs` instead of `hidden[-1]`), explains that the LSTM itself was already computing every hidden state in both cases, and shows why applying the same classifier head to every hidden state individually, rather than once to a single summary, is what produces per-token rather than per-sequence predictions.