Pseudoscience & the Demarcation Problem in Practice

Scientific Methodology

Chapter 7 · Pseudoscience & the Demarcation Problem in Practice

Chapter 1 introduced Popper's falsifiability criterion; Chapter 2 introduced the real legal test applied in Kitzmiller v. Dover. This chapter applies both tools to real, contested cases — including one genuinely surprising fact: the demarcation problem isn't only a "science vs. obvious pseudoscience" question. It's actively, publicly argued over inside mainstream physics itself.

A Real, Tested Case: Homeopathy

Homeopathy is a useful first case precisely because it is not simply unfalsifiable — it makes specific, testable claims (this remedy improves this condition compared with placebo), and those claims have been tested, repeatedly, at real scale.

Australia's NHMRC, March 2015

The National Health and Medical Research Council examined 57 systematic reviews covering 176 individual studies, across 61 health conditions, using only controlled trials. Its real, precise conclusion: "there are no health conditions for which there is reliable evidence that homeopathy is effective."

A second, independent demarcation angle comes from homeopathy's own preparation method. Remedies are made by repeated serial dilution — often labelled "X" (1:10 per step) or "C" (1:100 per step). Past roughly 12C dilution (about 1 part in 1024), Avogadro's number (~6.022 × 1023) means there is a vanishingly small real probability that even a single molecule of the original substance remains — yet many commercial remedies are diluted to 30C or higher.

Why This Case Matters for Demarcation
Homeopathy isn't disqualified because it can't be tested — it's disqualified because it was tested, repeatedly, and failed. When the dilution problem is raised, proponents have proposed new mechanisms in response ("water memory," more recently claims about water's own "coherent domains") — each one untested and each one arriving only after the previous explanation was challenged. That pattern of continually inventing a new, unfalsified rescue has its own name, covered next.

The Demarcation Problem Inside Mainstream Science: String Theory

A genuinely striking real fact: whether a theory satisfies Popper's own falsifiability criterion is not only debated about pseudoscience — it is a live, unresolved dispute among working physicists, about one of the most prominent research programs in modern theoretical physics.

Lee Smolin, 2006

The Trouble with Physics. Argues string theory's "landscape" of an astronomical number of possible vacuum states (commonly cited around 10500) means it may accommodate almost any observed outcome — casting real doubt on what it actually forbids.

Peter Woit, 2006

Not Even Wrong — the title borrows physicist Wolfgang Pauli's real put-down for an idea "so incomplete that it could not even be used to make predictions to compare with observations." Woit argues string theory has produced no falsifiable predictions at all.

Sabine Hossenfelder

A more recent, vocal critic arguing string theory has never been used to derive the Standard Model's own real physical constants, and that its landscape raises doubt about whether it meaningfully constrains reality at all.

An Honest Complication
This is a real, unresolved dispute, not a settled verdict — plenty of working physicists still consider string theory a serious, legitimate research program, and the debate itself is ongoing. The point for this course isn't who's right; it's that the demarcation question genuinely applies at physics' own frontier, not just to fringe claims.

Ad Hoc Rescue: The Mars Effect

French psychologist and statistician Michel Gauquelin claimed a real statistical correlation between Mars's position at birth and later becoming an elite sports champion — the "Mars effect." Belgium's Comité Para tested the claim in 1967 and initially reported a replication, though most of the underlying data had still been collected by Gauquelin himself. Subsequent independent replication attempts largely failed.

The Real Rescue Maneuver

When the effect failed to replicate cleanly, defenders introduced new post hoc statistical constructs — an "eminence test" and an "IMQ bias indicator" — that effectively redefined, after the fact, which athletes counted as sufficiently notable for the effect to apply to. These constructs were tuned to fit the existing data rather than specified in advance, which is exactly what makes them an ad hoc rescue rather than a genuine, independent confirmation.

This is the pattern to watch for generally: a specific, testable claim fails a fair test, and rather than the claim being abandoned or revised in a way that could itself be tested again, a new explanation is invented specifically to explain away that one failure — with no independent evidence of its own.

Why Vague Claims Feel Confirmed Anyway: The Forer Effect

In 1948, psychologist Bertram Forer gave 39 of his students a personality test, telling each they would receive a personalized write-up. Every student actually received the identical, generic description, assembled from a newspaper astrology column. Asked to rate its accuracy, the average score was 4.26 out of 5 — before the trick was revealed. Psychologist Paul Meehl later named this the "Barnum effect" (1956); it's often called the Forer effect today. It doesn't explain the Mars effect's specific failure above, but it explains why astrology's broader appeal survives such failures — vague, universal statements feel personally accurate to nearly everyone.

Popper's Own Original Target: Psychoanalysis

Popper's falsifiability criterion has a real, specific origin: in the early 1920s he observed in Alfred Adler's Vienna clinics, and contrasted his growing skepticism of psychoanalysis with his real admiration for Arthur Eddington's 1919 solar-eclipse test of general relativity — a theory that made one specific, risky, falsifiable prediction that could have failed, and didn't (Chapter 1).

"As for Freud's epic of the Ego, the Super-ego, and the Id, no substantially stronger claim to scientific status can be made for it than for Homer's collected stories from Olympus. These theories describe some facts, but in the manner of myths... not in a testable form."
— Karl Popper, Conjectures and Refutations

Popper argued psychoanalysis was "simply non-testable, irrefutable" — that there was "no conceivable human behaviour which could contradict" it.

A Real Complication, Even Here
Philosopher Adolf Grünbaum later argued Popper's charge went too far — that psychoanalytic claims genuinely are testable in principle, but that specific predictions mostly fail when actually tested. That's a different, and in some ways harsher, criticism than pure unfalsifiability — a reminder that even a founding demarcation case isn't unanimously settled among philosophers of science.

Putting the Cases Side by Side

CaseMakes testable claims?Tested?Result
HomeopathyYesYes, extensively (NHMRC 2015)No reliable evidence found, across 61 conditions
String theoryDisputedNot yet, per criticsUnresolved — a live dispute inside physics itself
Mars effectYesYes (Comité Para, 1967 onward)Failed replication; rescued with post hoc statistical constructs
Psychoanalysis (Popper's charge)Popper: noGrünbaum: yes, and it mostly failsGenuinely disputed even among philosophers of science

Hands-On Exercises

Exercise 1

Explain, in your own words, why the homeopathy case is a stronger illustration of the demarcation problem than a case that's simply unfalsifiable from the start — what does the NHMRC review actually demonstrate that a pure falsifiability critique alone couldn't?

📄 View solution
Exercise 2

Explain, in your own words, why the real string theory dispute shows the demarcation problem isn't confined to fringe claims — and what would have to happen for the dispute to actually be resolved one way or the other.

📄 View solution
Exercise 3

Explain, in your own words, precisely what made the Mars effect's "eminence test" and "IMQ bias indicator" an ad hoc rescue rather than a legitimate methodological refinement — what would have made it legitimate instead?

📄 View solution

Chapter 7 Quick Reference

  • Homeopathy: NHMRC's 2015 review of 57 systematic reviews/176 studies/61 conditions found no reliable evidence — a testable claim that was tested and failed, not an unfalsifiable one
  • String theory's falsifiability is a real, unresolved dispute inside mainstream physics itself (Smolin, Woit, Hossenfelder)
  • The Mars effect failed independent replication and was rescued with new, post hoc statistical constructs — the ad hoc rescue pattern
  • The Forer effect (1948) explains why vague personality claims feel personally accurate to nearly everyone
  • Popper's own original demarcation target was psychoanalysis — though even that verdict is disputed by philosopher Adolf Grünbaum