Exercise 3: Why Knowledge Cutoff Is Structural, and Why RAG Is a Workaround Not a Fix — Possible Solution ==================================================================== WHAT PRETRAINING ACTUALLY ENCODES, PER llm1-7 ------------------------------ Per llm1-7, pretraining uses a corpus collected up to some specific point in time, and the model's own knowledge afterward is "entirely encoded in its trained parameters, shaped exclusively by whatever text existed in that corpus." Once training finishes, the parameters are fixed — they don't continue changing as new events happen in the world. WHY THIS MAKES KNOWLEDGE CUTOFF A STRUCTURAL, NOT INCIDENTAL, PROPERTY ------------------------------ Per this chapter, "nothing about the pretraining process gives the model any live, ongoing access to information published after its own training data was collected — there is no built-in internet connection, no update mechanism running in the background." This isn't a limitation someone chose to impose; it follows directly from what pretraining actually is — a one-time process that shapes parameters based on a fixed dataset, with no mechanism anywhere in llm1-2 through llm1-9's own pipeline for continuously ingesting new information after that process ends. WHY MISSING POST-CUTOFF INFORMATION ISN'T "UNAWARENESS" IN ANY VAGUE SENSE ------------------------------ Per this chapter, "anything that happened after the corpus's own collection date is simply absent from the patterns the model learned — not because the model is vaguely 'unaware,' but because that information never existed in any data used to shape its parameters at all." This is a precise, mechanical claim: the model's parameters were never exposed to that information during training, so there was never any opportunity for a pattern involving it to be learned in the first place. It isn't that the model forgot or overlooked something; the information was structurally absent from the entire training process. WHY RETRIEVAL-AUGMENTED GENERATION IS DESCRIBED AS A WORKAROUND ------------------------------ Per this chapter, RAG "works around this by injecting live external information directly into the prompt at inference time — a way of supplying missing context, not a way of updating the model's own trained parameters." When RAG is used, the model's own weights are completely unchanged — it's still the identical pretrained-and-fine- tuned model from llm1-9. What changes is the input: relevant, current information is retrieved from an external source and placed directly into the prompt itself, so the model can use it the same way it uses any other context provided at inference time. WHY THIS DISTINCTION MATTERS ------------------------------ Because RAG never touches the model's own parameters, it doesn't extend what the model "knows" in the sense llm1-7 defined knowledge — it simply gives the model access to specific external information for that particular conversation, exactly once, through the prompt. This is why RAG is accurately described as compensating for the knowledge- cutoff limitation from outside the model, rather than actually removing the underlying structural cause of it. WHY THIS WORKS AS AN ANSWER ------------------------------ It traces knowledge cutoff back to the fixed, one-time nature of pretraining described in llm1-7, explains precisely why missing information reflects structural absence from training data rather than vague unawareness, and explains why RAG's own mechanism (injecting information into the prompt, not updating parameters) makes it a workaround rather than a genuine fix for the underlying cause.