Hypothesis Space Partitioning¶

Why It Matters¶

Informal reasoning often evaluates one hypothesis at a time ("is X true?") without considering alternatives. This generates false binaries, ignores unlikely-but-important possibilities, and makes belief updating incoherent (what do we update toward if not-X?). Partitioning forces completeness: you must account for all possibilities and their relative plausibilities. This reveals unstated assumptions and hidden hypotheses.

The Key Move¶

When facing uncertainty, write down a list of distinct hypotheses that could be true. Check: are they mutually exclusive (at most one is true)? Are they exhaustive (at least one is true)? If not, split or merge until they partition the space. Assign each a prior probability; they must sum to 100%. As evidence arrives, update all hypotheses simultaneously. This disciplines thinking by preventing: evaluating hypotheses in isolation, forgetting alternatives, or sneaking in assumption that unexamined hypotheses are impossible.

Classic Application¶

Medical differential diagnosis. Patient presents with fever and rash. Hypotheses: viral infection (60%), bacterial infection (25%), allergic reaction (10%), autoimmune condition (4%), other (1%). These partition the space - exactly one explains the symptoms. Each new test (evidence) updates all probabilities simultaneously. The "other" category prevents overconfidence by acknowledging unknown unknowns. The framework forces: consider all plausible causes, quantify relative likelihood, update coherently.

Surprising Application¶

Debugging mysterious software failures. Bug could be in: new code (40%), dependency update (30%), infrastructure change (20%), race condition now exposed (8%), other (2%). Partitioning prevents "it must be in my code" tunnel vision. As each hypothesis is tested (add logging, rollback dependency, check infrastructure), update all probabilities. The "other" category keeps you humble when all obvious explanations fail. Prevents wasting time re-checking hypotheses you've already ruled out.

Failure Modes¶

False precision in partition boundaries - reality is continuous but partitions are discrete. Leaving "other" category too small - when unknown unknowns dominate, the partition is misleading. Computationally explosive - fine-grained partitions (many hypotheses) become intractable. Unstable under new evidence - sometimes evidence suggests hypotheses you hadn't imagined, breaking the partition.

Go Deeper¶

Jaynes, Probability Theory: The Logic of Science, Chapter 2 on Boolean algebra of propositions; Pearl, Probabilistic Reasoning in Intelligent Systems, Chapter 2.