Exercise 2: The Mechanical Reason Hallucination Happens — Possible Solution ==================================================================== WHAT THE MODEL WAS ACTUALLY TRAINED TO DO, PER llm1-7 ------------------------------ Per llm1-7's own causal language modeling objective, the model is trained at every position to predict "the token that actually comes next, given everything before it" — measured purely by how well the predicted probability distribution matches the token that genuinely followed in the training corpus. Nowhere in this objective is there any separate signal for "is this statement true," only "is this the most statistically probable continuation given the preceding context." WHY THIS MEANS THE MODEL HAS NO BUILT-IN TRUTH-CHECKING STEP ------------------------------ Because the entire training signal is about predicting plausible continuations, not verifying facts, nothing in the model's own core architecture (llm1-4's attention, llm1-5's Transformer blocks) performs any kind of fact-checking or truth-verification computation. The model produces whatever continuation its learned parameters assign the highest probability to, and that probability reflects patterns in the training data, not an independent check against reality. WHAT HAPPENS WHEN THE MODEL HAS WEAK PATTERN SUPPORT FOR THE TRUE ANSWER ------------------------------ Per this chapter, "when the model has no strong pattern support for the true continuation, its core mechanism has no fallback behavior for that case — it simply continues producing some plausible-sounding sequence of tokens." The generation process (per llm1-6's own causal decoding) doesn't pause or refuse when confidence is low; it still produces a probability distribution over the vocabulary and samples or selects from it, the same way it does when confidence is high. The resulting tokens can be fluent and plausible-sounding while being factually wrong, because fluency and plausibility, not truth, are exactly what the training objective optimized for. WHY LEARNED "I DON'T KNOW" RESPONSES DON'T CONTRADICT THIS ------------------------------ Per this chapter, llm1-9's own RLHF pipeline "can and does train a model to say 'I don't know' more often in situations where that response is reward-preferred." This doesn't mean the base architecture gained a genuine fact-checking capability — it means the reward model (llm1-9) was trained on human preferences that favored honest uncertainty in certain recognizable situations, and RLHF then made "I don't know"-style continuations more probable in those specific situations. This is a learned behavioral pattern layered on top of the same underlying mechanism, not a new core capability — the model still has no independent way to verify truth; it has simply learned, via reward signal, that admitting uncertainty is sometimes the higher-reward continuation to produce. WHY THIS WORKS AS AN ANSWER ------------------------------ It traces hallucination back to the exact objective llm1-7 trained the model on, explains why nothing in that objective performs fact- verification, shows what the generation mechanism does by default when confidence is low, and explains precisely why RLHF-trained uncertainty responses are a learned behavioral layer rather than evidence of a built-in fact-checking mechanism.