Exercise 2: How the File Drawer Problem Biases Even an Honest Review — Possible Solution =========================================================================================== The bias described by Rosenthal's file drawer problem happens before the review even begins - it's baked into which studies exist in published form to be found at all, not into how the reviewer selects among them afterward. Studies that find a significant, positive effect are more likely to be written up, submitted, and accepted for publication. Studies that find no effect are more likely to go unpublished entirely, sitting unreported in a researcher's own files (the literal "file drawer" Rosenthal's own term refers to). A reviewer searching the published literature exhaustively and in complete good faith will still, structurally, encounter a pool of studies that already overrepresents positive findings relative to the true underlying reality - simply because the negative and null-result studies were never published for the reviewer to find in the first place. This is a genuinely different problem from cherry-picking. Cherry- picking happens when a reviewer selectively chooses among available studies. The file drawer problem happens even when a reviewer makes no selective choice at all, because the available studies themselves are already a skewed sample of everything that was actually studied. ANSWER: The file drawer problem biases the underlying pool of published studies itself, not the reviewer's own selection process - studies with positive results are more likely to be published, while null-result studies are more likely to remain unpublished. A reviewer reading every available published study in good faith is still working from a systematically skewed sample, since the true population of studies conducted (including the unpublished negative ones) was never fully represented in what was available to review. WHY THIS WORKS AS AN ANSWER ------------------------------ This correctly locates the bias in the published literature itself rather than in reviewer behavior, distinguishing it from cherry- picking as the exercise specifically asked.