Evaluating Source Credibility

Research Methodology

Chapter 4 · Evaluating Source Credibility

Chapter 3 ended on a real gap: knowing a source is primary doesn't mean it's trustworthy. This chapter covers a real, established framework for closing that gap, how peer review actually works (and where it genuinely fails), and a documented case of an entire tier of "peer-reviewed" journals that will publish almost anything for a fee.

The CRAAP Test

Real, Documented Origin

Librarian Sarah Blakeslee coined the CRAAP acronym in 2004 while developing a workshop for first-year students at California State University, Chico. It's since spread well beyond that one campus into information-literacy instruction worldwide.

Currency

How recent is it, and does the topic actually need current information?

Relevance

Does it actually address your specific question, at an appropriate depth?

Authority

Who wrote or published it, and what are their real, checkable qualifications?

Accuracy

Is it supported by evidence, and can it be verified elsewhere?

Purpose

Why does this source exist — to inform, persuade, sell, or entertain?

How Peer Review Actually Works

A submitted paper is first screened by a journal editor, then sent to independent experts in the same field who evaluate its methods, evidence, and conclusions before publication is approved. It's a real, genuinely useful filter — but Chapter 1's own Wakefield case is a direct reminder that peer review checks whether a paper's stated methods are internally consistent, not whether its underlying data or funding sources are honest. A reviewer generally can't independently re-verify a researcher's own raw records.

Predatory Journals: A Real, Documented Trap

University of Colorado Denver librarian Jeffrey Beall coined the term "predatory publishing" and maintained a real, widely used list of predatory open-access publishers from 2010 until he removed it in January 2017, following legal threats and formal complaints from publishers named on it. These are outlets that charge authors a publication fee while providing minimal or no genuine peer review.

A Real, Measured Sting — Same Journalist as Chapter 3

In October 2013, journalist John Bohannon — the same reporter behind the 2015 chocolate hoax in Chapter 3 — published "Who's Afraid of Peer Review?" in Science. He submitted a deliberately flawed fake paper, claiming a lichen extract as a cancer cure, to 304 open-access journals. 157 accepted it, 98 rejected it, and the rest were still under review or unresponsive. Whenever a journal accepted the paper, Bohannon withdrew it before it entered the real scientific record.

An Honest Limitation of the Sting

Bohannon's own study submitted the fake paper only to open-access journals, with no subscription-journal control group — a real, documented critique of the sting's design. It demonstrates that predatory open-access journals accept fatally flawed papers at an alarming real rate; it doesn't, on its own, prove subscription journals would have done any better.

Legitimate vs. Predatory: Real Warning Signs

Legitimate journal	Predatory journal
Editorial board is real, named, and reachable	Editorial board is fake, incomplete, or unresponsive
Fees, if any, are disclosed clearly upfront	Fees appear only after acceptance
Indexed in recognized real databases	Claims impressive-sounding but unverifiable indexing
Genuine, substantive peer review	Acceptance within days, little or no real feedback

Hands-On Exercises

Exercise 1

Apply all five CRAAP criteria to a specific article you've recently read online (news, blog, or otherwise). For each criterion, write one sentence explaining whether the source passes or fails it, and why.

📄 View solution

Exercise 2

A classmate argues that Bohannon's 2013 sting proves peer review is worthless everywhere. Using the chapter's own "honest limitation" note, explain why this conclusion overreaches what the real study actually showed.

📄 View solution

Exercise 3

Using the "Real Warning Signs" table, list three specific, checkable things you could look for on a journal's own website within five minutes to judge whether it's likely predatory.

📄 View solution

Chapter 4 Quick Reference

The CRAAP test (Currency, Relevance, Authority, Accuracy, Purpose) was coined by Sarah Blakeslee at CSU Chico in 2004
Peer review checks internal consistency, not underlying data honesty — it did not catch Wakefield's fraud from Chapter 1
Jeffrey Beall coined "predatory publishing" and maintained a real list of predatory journals from 2010 to 2017
John Bohannon's real 2013 Science sting found 157 of 304 open-access journals accepted a deliberately flawed fake cancer-cure paper
The sting's own honest limitation: no subscription-journal control group, so it can't directly compare predatory and legitimate journals' acceptance rates