Likelihood Ratio Thinking¶

Why It Matters¶

People confuse "evidence for A" with "A is true." But evidence strength and conclusion probability are distinct. You can have strong evidence (high likelihood ratio) for an improbable hypothesis that remains improbable after updating. Conversely, weak evidence (likelihood ratio near 1) for a very probable hypothesis doesn't make the hypothesis uncertain. Likelihood ratio thinking cleanly separates "what does this evidence tell us?" from "what should we believe?"

The Key Move¶

For any piece of evidence, ask: "How probable is this observation if hypothesis A is true? If hypothesis B is true?" Calculate or estimate the ratio. Ratios of 3-10 are moderate evidence; 10-100 are strong; over 100 are very strong. Ratios between 0.3-3 are weak evidence that barely matters. This quantifies "how diagnostic is this evidence?" independent of prior beliefs. Then, and only then, combine with priors to reach conclusions.

Classic Application¶

Forensic DNA evidence. A DNA match has an astronomical likelihood ratio (maybe 1 billion to 1) - the match is vastly more likely if the suspect is the source than if they're not. This is strong evidence. But if the prior probability the suspect is guilty is very low (maybe they're one of millions in a database), even a billion-to-one likelihood ratio might only raise posterior guilt probability to moderate levels. The likelihood ratio correctly captures evidence strength without confusing it with guilt probability.

Surprising Application¶

Evaluating advice from experts with known biases. An expert known to oversell their domain recommends their approach. The likelihood ratio is near 1 - they'd recommend this whether or not it's optimal. This is weak evidence despite coming from an expert. Conversely, a skeptical expert endorsing a surprising approach has a high likelihood ratio - they wouldn't say this unless evidence was strong. The tool lets you quantify expert credibility as evidential weight, not authority.

Failure Modes¶

Denominator neglect - focusing only on P(evidence|hypothesis) without considering P(evidence|alternatives). Alternative hypothesis vagueness - the ratio requires specific alternatives; "all other possibilities" is often ill-defined. Independence assumptions - likelihood ratios only combine multiplicatively if evidence pieces are independent given hypotheses. Mistaking direction - a likelihood ratio of 0.1 is moderate evidence against, not weak evidence.

Go Deeper¶

Royall, Statistical Evidence: A Likelihood Paradigm, Chapter 1; Good, "Weight of Evidence: A Brief Survey" (Bayesian Statistics, 1985).