Exercise 2: Identifying Type I and Type II Errors — Possible Solution ==================================================================== THE GIVEN SCENARIO: TYPE I ERROR ------------------------------ The team rejected H0 (concluded the feature had a real effect) when, in reality, H0 was actually true (no real effect existed - the observed difference was just sampling noise). Per this chapter's own 2x2 table, rejecting H0 when H0 is actually true is precisely the definition of a TYPE I ERROR - a false positive. The team's significance threshold (alpha) was exactly the mechanism that was supposed to control how often this happens, but a Type I error is still possible any time a test is run, purely by chance, at whatever rate alpha allows. A SECOND SCENARIO SHOWING THE OPPOSITE ERROR: TYPE II ERROR ------------------------------ Suppose a different team tests a new checkout flow that genuinely does improve conversion rate in reality (H0 is actually false - there IS a real effect). However, their test sample size was small, or the true effect size was modest, so the resulting p-value came out above alpha (say, p = 0.11), and the team failed to reject H0, concluding "no significant effect found" and discarding a genuinely beneficial feature. Per this chapter's own 2x2 table, failing to reject H0 when H0 is actually false is precisely a TYPE II ERROR - a false negative, a real effect that went undetected. WHY THIS WORKS AS AN ANSWER ------------------------------ The given scenario is matched directly to this chapter's own 2x2 error table by identifying which cell it falls into (reject H0 / H0 actually true), and the second scenario is deliberately constructed to land in the opposite cell (fail to reject H0 / H0 actually false), demonstrating a clear understanding of both error types rather than just naming one and describing the other only vaguely.