Skip to content

Precision vs. Recall Tradeoff

Why It Matters

Different problems have radically different cost structures. Missing a disease (false negative) might be catastrophic while a false alarm (false positive) is merely inconvenient - prioritize recall. Falsely flagging normal email as spam (false positive) loses important messages while missing spam (false negative) is tolerable - prioritize precision. Most ML tutorials default to balancing precision and recall, which is optimal for almost no real problem.

The Key Move

Before evaluating a classifier, explicitly determine: What is the cost of a false positive? What is the cost of a false negative? Are they equal (rare)? Set decision thresholds to minimize expected cost, not to balance precision and recall. Plot the precision-recall curve to see the full tradeoff space, then choose the operating point that matches your cost structure.

Classic Application

Airport security screening. False negative (missing a threat) has enormous cost; false positive (extra screening) has small cost. Rational approach: accept low precision (many false alarms) to achieve high recall (catch nearly all threats). The opposite tradeoff would be catastrophic but look better on standard metrics.

Surprising Application

Job hiring processes. False positive (hiring a poor fit) has high cost - wrong hires damage teams and are hard to remove. False negative (rejecting a good candidate) has lower cost - you'll fill the role with someone else. This justifies high precision, low recall: rigorous screening even if you miss some good candidates. Many companies optimize the wrong metric.

Failure Modes

Unknown costs - without explicit cost structure, you can't choose rationally among tradeoffs. Dynamic costs - cost ratios change over time or context, invalidating fixed thresholds. Multi-class confusion - precision/recall are binary concepts; multi-class problems require more sophisticated metrics.

Go Deeper

Davis & Goadrich (2006), "The Relationship Between Precision-Recall and ROC Curves," ICML; Flach (2016), "ROC Analysis," Encyclopedia of Machine Learning

  • Screening - Information Economics: Reasoning Under Asymmetry and Uncertainty