Skip to content

Multiple Measurement

Why It Matters

Every measurement has error - systematic bias, random noise, or both. Relying on a single measure means you can't distinguish true signal from measurement artifact. Multiple measures with different error structures allow you to identify the common signal. Convergence across methods is strong evidence; divergence is a crucial warning that your interpretation might be wrong.

The Key Move

For any important variable, plan at least 2-3 independent measurement approaches. Combine self-report with behavioral observation with physiological measures. Use multiple raters. Vary question phrasing. Check if the pattern holds across different operationalizations. If measures disagree substantially, investigate why rather than cherry-picking the one that supports your hypothesis.

Classic Application

Construct validation in psychology. To measure anxiety, combine

  1. self-report questionnaires,
  2. behavioral coding of nervous behaviors,
  3. physiological measures like heart rate and cortisol, and
  4. observer ratings. Convergence across these different methods strengthens confidence that you're measuring anxiety rather than response bias, demand characteristics, or artifact.

Surprising Application

Evaluating employee performance. Don't rely solely on manager ratings (one method, vulnerable to recency bias and favoritism). Add: peer ratings, objective output metrics, customer feedback, self-assessment, and 360-degree review. Divergence is informative - someone rated high by managers but low by peers signals something important about organizational dynamics.

Failure Modes

Using "multiple measures" that are actually the same method (three self-report questionnaires aren't independent). Averaging measurements without checking for convergence (hides important disagreement). Ignoring divergence and reporting only the converging measures. Treating convergence as proof of validity rather than as supporting evidence.

Go Deeper

Campbell & Fiske (1959), "Convergent and discriminant validation by the multitrait-multimethod matrix," Psychological Bulletin 56(2); Webb et al. (1966), Unobtrusive Measures: Nonreactive Research in the Social Sciences.