Exercise 3: Why Knowing Which Feature Matters More Doesn't Tell You If the Model Is Good — Possible Solution ==================================================================== WHAT THIS CHAPTER'S OWN FINDING ACTUALLY ESTABLISHES ------------------------------ Per this chapter's own finding-box, the conclusion reached is specifically about RELATIVE importance: "mileage is the stronger predictor of the two, holding year constant." This is a comparison between two coefficients' own relative sizes — it says which of the two features pulls harder on the prediction, nothing more. WHY THIS SAYS NOTHING ABOUT OVERALL PREDICTION QUALITY ------------------------------ Per this chapter, "a model could correctly rank mileage as the stronger factor while still being wildly inaccurate in its actual dollar predictions." Consider a model whose predicted prices are consistently off by tens of thousands of dollars from the real values across the board — it could still, correctly, show mileage's own coefficient as larger than year's, since that comparison only concerns the RELATIVE weight the model assigns each feature, completely independent of how close the model's own predicted numbers actually land to the real prices. A model can be "right about which feature matters more" and "bad at prediction" at the same time, because these are two entirely different properties being measured. WHY THESE ARE GENUINELY SEPARATE QUESTIONS ------------------------------ "Which feature matters more" is a question about the model's own internal structure — the relative sizes of its learned coefficients. "How good are this model's actual predictions" is a question about the model's OUTPUT compared against real, known answers — something this chapter never actually checked. Nothing in fitting the model or reading off its coefficients requires comparing any predicted price against an actual price at all, which is exactly why the coefficients alone can't answer the second question. WHAT WOULD ACTUALLY BE NEEDED TO ANSWER THE SECOND QUESTION ------------------------------ Per this chapter's own closing line, "ml1-4 covers exactly that next." Answering "how good are the predictions" requires comparing the model's own predicted prices against the real prices held out in the test set (ml1-2's own train/test split), using dedicated metrics built specifically for that comparison — the exact material ml1-4 introduces (MAE, MSE, RMSE, R²) — none of which this chapter computed or discussed at all. WHY THIS WORKS AS AN ANSWER ------------------------------ It distinguishes precisely between what this chapter's own finding measures (relative coefficient size) and what "is the model any good" actually requires (comparing predictions against real held-out values), explains why a model can score well on one while failing the other, and identifies ml1-4's own evaluation metrics as the missing piece.