Exercise 3: Why Randomization Rules Out Confounding — Possible Solution ==================================================================== THE PROBLEM WITH OBSERVATIONAL CORRELATION ------------------------------ When two variables are simply observed together "in the wild" (like CPU usage and response time, or ice cream sales and drownings), any number of unmeasured, unknown confounders could be silently driving both at once. There's no way to rule this out just by looking at the correlation itself - per this chapter's own four-explanation framework, a confounder remains a live possibility no matter how strong the observed correlation is. WHY RANDOM ASSIGNMENT FIXES THIS ------------------------------ Per this chapter's own finding, randomly assigning users to control or treatment (Chapter 6's own A/B test) guarantees, BY CONSTRUCTION, that any potential confounder - traffic source, time of day, device type, user tenure, anything at all, including confounders nobody even thought to measure - gets distributed roughly equally between the two groups. This isn't something that has to be verified case by case; it follows directly from the mechanics of random assignment itself, since a coin flip (or equivalent) has no way to "know about" or systematically favor one group with more of any particular confounding trait. WHY THIS IS FUNDAMENTALLY DIFFERENT FROM OBSERVATIONAL DATA ------------------------------ With purely observational data, the groups being compared were never randomly formed - users who behave one way and users who behave another way may differ in countless unmeasured respects that a researcher can't see or control for. With random assignment, the two groups start out statistically equivalent on every dimension, measured or not, purely due to the randomization process itself. If the outcome still differs significantly afterward, the treatment is the only remaining systematic difference between the groups that could explain it - confounding has been ruled out by design, not by hoping none exists. WHY THIS WORKS AS AN ANSWER ------------------------------ The explanation is grounded directly in this chapter's own finding about randomization, specifically emphasizing that it works even against confounders nobody has thought to measure - the key distinction that separates a randomized experiment from any amount of purely observational correlation, however strong.