Exercise 1: Tracing the Capstone's Own Code Back to Its Source Chapters — Possible Solution ==================================================================== TRACING EACH STEP TO ITS ORIGINATING CHAPTER ------------------------------ `tokens = [...]` and `token_ids = [...]` — llm1-2's own Byte-Pair Encoding, converting the raw prompt string into subword tokens and then integer vocabulary IDs, with no out-of-vocabulary risk thanks to that chapter's own byte-level fallback. `x = embedding_table[token_ids]` followed by `x = x + positional_encoding[0:5]` — llm1-3's own token embedding lookup, combined with positional encoding, resolving nlp1-8's own deferred order problem before any attention runs. `for layer in range(num_layers): x = x + multi_head_attention(...); x = x + feed_forward(...)` — llm1-5's own full Transformer block, built from llm1-4's own Query/Key/Value scaled dot-product attention, with the `causal_mask=True` argument specifically llm1-6's own contribution. `logits = final_layer_output[-1] @ output_projection` and `probabilities = softmax(logits)` — llm1-7's own next-token prediction objective, made reliable at real scale per llm1-8's own scaling laws. The repeat-the-whole-process description in Step 5 — llm1-6's own autoregressive generation loop, only possible because causal masking already guarantees consistency between training and generation. WHY llm1-1's OWN VERSION WAS DELIBERATELY A "PREVIEW," NOT A TRACE ------------------------------ Per llm1-1, that chapter described the same "capital of France" example only in general terms — "broken into pieces," "converted into a vector," "pass through many stacked layers" — without any of the actual mechanisms (BPE's merge algorithm, Q/K/V matrices, causal masking, the training objective) having been built yet. It was accurate in outline but couldn't be verified or examined mechanically, since none of the underlying pieces existed in the reader's own understanding at that point in the course. WHY THIS CHAPTER'S OWN VERSION IS A GENUINE MECHANICAL ACCOUNT ------------------------------ Every line in this chapter's own trace corresponds to a specific, already-explained mechanism from a specific earlier chapter — nothing here is asserted without a traceable origin. Where llm1-1 could only say "the pieces pass through layers that let every piece attend to every other piece," this chapter can say precisely what that means: Query/Key/Value projections (llm1-4), computed per attention head (llm1-5), restricted by a causal mask (llm1-6), for reasons made concrete in the attribution table. WHY THIS WORKS AS AN ANSWER ------------------------------ It maps each line of this chapter's own trace to the specific chapter responsible for it, and explains precisely what distinguishes llm1-1's own deliberately abstract preview from this chapter's own fully traceable mechanical account — namely, that every step here can be verified against a mechanism that was actually built and explained earlier in the course.