Why Critical Thinking Is a Trainable Skill

Critical Thinking

Chapter 1 · Why Critical Thinking Is a Trainable Skill

"Some people are just naturally better thinkers" is a genuinely popular belief — and a real, rigorous four-year experiment involving over a million forecasts found it doesn't hold up. This course covers critical thinking as a set of specific, learnable habits, not a fixed personal trait, starting with the real study that proved it.

A Real, Rigorous Test

The Good Judgment Project, 2011-2015

In 2011, IARPA — the US intelligence community's own research arm — launched a real forecasting tournament to find the most accurate methods for predicting geopolitical events. Philip Tetlock, Barbara Mellers, and Don Moore's Good Judgment Project entered ordinary volunteers, given some basic probabilistic-reasoning training, against professional intelligence analysts with access to classified information. Over four years, across 500 real questions and more than a million individual forecasts, the volunteer forecasters — nicknamed "superforecasters" — won decisively.

The Real, Measured Result
The top-performing volunteers outperformed professional intelligence analysts — people with formal training and classified access these volunteers never had — by more than 30% in forecasting accuracy. Whatever separated the best forecasters from everyone else wasn't formal credentials, security clearance, or innate talent — it was a specific, describable set of habits.

What They Actually Did Differently

  • Fermi-izing: breaking one large, hard-to-answer question into smaller sub-questions that are each individually more tractable — e.g., not "will this conflict escalate," but "what's the probability of the precursor event, and given that, what's the probability of escalation."
  • Actively open-minded thinking: deliberately seeking out sources and views that disagreed with their own initial estimate, and taking the strongest version of the opposing case seriously — treating a belief as a hypothesis to test, not something to defend.
  • Incremental updating: revising a forecast in small steps as new evidence arrived, rather than swinging dramatically between extremes or refusing to update at all once a first impression had formed.

What This Course Actually Covers

2

Fallacies

Recognizing flawed argument structures

3

Cognitive Biases

Anchoring, availability, Dunning-Kruger

4

Correlation & Causation

Including Simpson's Paradox

5

Statistical Traps

Base rates, the prosecutor's fallacy

6

Propaganda

Persuasion techniques, the illusory truth effect

7

Framing

How wording alone shifts a decision

8

Groupthink

Decision-making under real pressure

9

A Practical Checklist

Applying it to everyday claims

10

Capstone

Deconstructing a real, complex argument

Hands-On Exercises

Exercise 1

Pick a real prediction you've made recently (about a sports result, an election, a work deadline, anything). Apply Fermi-izing: break it into two or three smaller sub-questions that would each be easier to estimate individually than the original question was as a whole.

📄 View solution
Exercise 2

Explain, in your own words, why the Good Judgment Project's result is more convincing evidence that critical thinking is trainable than simply pointing to one smart person who happens to think clearly.

📄 View solution
Exercise 3

Describe a real belief or opinion you currently hold. Explain what "actively open-minded thinking" would actually require you to go and do about that specific belief, concretely — not in general terms.

📄 View solution

Chapter 1 Quick Reference

  • The real Good Judgment Project (2011-2015) tested trained volunteer forecasters against professional intelligence analysts with classified access, across 500 questions and over a million forecasts
  • The volunteer "superforecasters" won, outperforming the professionals by more than 30%
  • Three real, specific techniques separated top performers: Fermi-izing (breaking questions into sub-questions), actively open-minded thinking (seeking disagreement), and incremental updating
  • None of these techniques required formal credentials or classified access — they're learnable habits, not fixed traits