Confidence Degradation Tracking¶

Why It Matters¶

People treat predictions as binary (either we know or we don't) when confidence actually decays continuously. This leads to using 10-day forecasts the same way as 2-day forecasts, or treating a 5-year business plan with the same confidence as next quarter's budget. Confidence degradation tracking reveals the "predictability horizon" - the point where your model adds no value over base rates.

The Key Move¶

For any prediction, ask: how does my confidence change as I project further from my known starting point? Mark explicit confidence intervals at different time horizons. Identify the timescale where uncertainty exceeds the signal (the "decorrelation time"). Beyond that horizon, revert to climatology/base rates rather than pretending your mechanistic model still works. Graph confidence as a function of time to make the degradation visible.

Classic Application¶

Weather forecasting skill. Meteorologists have 80-90% accuracy at 3 days, 70-80% at 5 days, barely better than climatology at 10 days, and no skill at 15 days. This isn't because they're lazy about the two-week forecast - it's fundamental chaos. The 10-day horizon is explicitly marked as the limit of deterministic predictability.

Surprising Application¶

Technology trend forecasting. When predicting tech adoption, confidence degrades based on generational cycles. Next-quarter hardware sales (short-term demand fluctuations) are somewhat predictable. Next-year platform shifts (single product cycles) have wider error bars. Five-year infrastructure changes (multiple product generations) are dominated by unpredictable breakthroughs. Marking these horizons prevents treating a 5-year tech roadmap with the same confidence as quarterly guidance.

Failure Modes¶

Linear extrapolation of confidence decay - confidence often drops nonlinearly, with sudden cliffs at critical timescales. Ignoring regime changes - confidence degradation rates change when the system enters new dynamics (calm to chaotic weather, peace to war). Treating the horizon as hard cutoff - predictability degrades continuously, not all at once. Over-anchoring on average skill - rare events may be more or less predictable than typical ones.

Go Deeper¶

Lorenz, "Deterministic Nonperiodic Flow," Journal of the Atmospheric Sciences (1963); Silver, The Signal and the Noise, Chapter 4.

Anchoring - Behavioral Economics: Reasoning About Systematic Deviations