Exercise 3: Why More Self-Selected Survey Responses Won't Fix a Biased Estimate — Possible Solution ==================================================================================== THE PROBLEM WITH THE SAMPLING METHOD ------------------------------ The company's survey only reaches customers who choose to fill out an optional online contact form - a small, self-selected group. Customers who bother to fill out an unprompted contact form are disproportionately likely to be either unusually satisfied (motivated to say something nice) or, in many real cases, unusually dissatisfied (motivated to complain) - either way, this specific method captures a systematically unrepresentative slice of the true customer population, not a representative cross-section of it. In this scenario, the resulting score being described as "very high" suggests the method is skewing toward the satisfied end. WHY THIS IS SAMPLING BIAS, NOT SAMPLING ERROR ------------------------------ Per this chapter's own distinction, sampling ERROR is the ordinary, expected wobble between different randomly-drawn samples of the same representative method - and that shrinks as more data is collected. Sampling BIAS is a flaw in the method itself, consistently excluding or over/under-representing part of the true population regardless of how many responses are gathered. This scenario is bias: the method never had a chance of reaching the large majority of customers who never think to fill out a contact form at all. WHY 10X MORE RESPONSES THROUGH THE SAME METHOD DOESN'T HELP ------------------------------ Per this chapter's own finding, collecting more data through a biased method only makes the ESTIMATE more precise, not more accurate - it tightens the sampling error around the same systematically wrong number, rather than correcting the number itself. Ten times more responses from the same self-selected group of contact-form users would produce an even more confidently-stated version of the same biased satisfaction score, not a more accurate one. Fixing this requires changing the sampling METHOD itself - for example, surveying a genuinely random cross-section of all customers, not just those who opt in - not simply collecting more responses through the existing flawed channel. WHY THIS WORKS AS AN ANSWER ------------------------------ The explanation identifies the specific sampling-bias mechanism at work (self-selection systematically excluding most customers), applies this chapter's own sampling-error-vs-sampling-bias distinction directly, and explains concretely why more data through the same method only sharpens the wrong answer rather than correcting it.