Conditional Independence Exploitation¶

Why It Matters¶

Real-world evidence comes in bundles - multiple symptoms, several test results, many data points. Reasoning about joint probabilities of dependent evidence is intractable. But if evidence pieces are conditionally independent given your hypotheses, you can treat them separately and multiply their likelihood ratios. This converts impossible calculations into manageable ones. The critical skill is recognizing when conditional independence holds and when it doesn't.

The Key Move¶

When you have multiple pieces of evidence, ask: "If I knew which hypothesis was true, would learning evidence A change my probability for evidence B?" If no, they're conditionally independent given that hypothesis - you can multiply their likelihood ratios. If yes, they're dependent - you must reason about them jointly or find a richer hypothesis space that explains the dependence. Draw a diagram: do the evidence pieces share a common cause (your hypotheses)? If the only connection is through the hypotheses, they're conditionally independent.

Classic Application¶

Naive Bayes text classification. A document contains words W1, W2, W3. Under category C (e.g., "spam"), these words appear with certain probabilities. The "naive" assumption: words are conditionally independent given the category. So P(W1, W2, W3 | spam) = P(W1|spam) × P(W2|spam) × P(W3|spam). This is computationally simple despite being literally false (words aren't independent). It works because conditional independence is approximately true: knowing a document is spam, word1 tells you little about word2 beyond what "spam" already told you.

Surprising Application¶

Evaluating multiple testimonies about an event. Three witnesses independently report seeing a suspect near the crime scene. If their testimonies are conditionally independent given guilt (one witness seeing the suspect doesn't make another more likely to, except through the common cause of the suspect actually being there), you can multiply their likelihood ratios. But if witnesses talked to each other, they're conditionally dependent - you must reason about their joint testimony. The tool reveals when "three independent witnesses" actually is independent evidence versus correlated stories.

Failure Modes¶

Assuming independence when it doesn't hold - the classic error of naive Bayes, which works in practice despite violating assumptions. Missing latent common causes - evidence pieces might be conditionally dependent given your hypotheses but independent given richer hypotheses you haven't considered. Confusing causal and probabilistic independence - variables can be causally related but probabilistically independent given knowledge of other variables. Over-application - sometimes dependent evidence still provides value; the tool isn't "ignore dependent evidence."

Go Deeper¶

Pearl, Causality, Chapter 1.2 on conditional independence; Murphy, Machine Learning: A Probabilistic Perspective, Chapter 2.2.4.