Skip to content

Pre-Registration and Transparency

Why It Matters

Researcher degrees of freedom are enormous: which variables to measure, which to control for, which outliers to exclude, when to stop collecting data, which analyses to report. If you analyze after seeing the data, you will find "significant" patterns by chance and confirmation bias. Pre-registration binds you to decisions made blind to results, distinguishing confirmatory from exploratory analysis. Transparency enables verification and prevents publication bias.

The Key Move

Before running any consequential experiment or analysis, write down:

  1. specific hypotheses with predicted direction,
  2. exact sample size and stopping rule,
  3. planned statistical tests and control variables,
  4. outlier exclusion criteria. Share this publicly (OSF, AsPredicted, etc.) with timestamp. After analysis, distinguish pre-registered confirmatory tests from exploratory post-hoc analyses. Share data and code.

Classic Application

Clinical trial registration (ClinicalTrials.gov). Before starting, researchers specify outcomes, sample size, and analysis plan. This prevents companies from running trials, getting unfavorable results, and selectively publishing only positive outcomes or switching to different outcome measures post-hoc. Dramatically improved medical research credibility.

Surprising Application

Personal experiments and self-tracking. Before trying a new habit or intervention, write down: exactly what you'll measure (and what you won't), how long you'll try it, what would count as success versus failure. After the experiment, you can't cherry-pick the outcomes that improved by chance. "I feel better" is post-hoc rationalization; "my pre-specified sleep quality metric improved by >15 minutes" is evidence.

Failure Modes

Pre-registering vague plans that allow extensive post-hoc flexibility. Conducting extensive exploratory analysis then presenting it as confirmatory. Running multiple studies and only pre-registering ones with good results. Treating pre-registration as bureaucratic compliance rather than epistemological commitment. Using transparency performatively (sharing data in unusable formats).

Go Deeper

Nosek, Ebersole, DeHaven, & Mellor (2018), "The preregistration revolution," Proceedings of the National Academy of Sciences 115(11); Miguel et al. (2014), "Promoting transparency in social science research," Science 343(6166).