Error Growth Tracking¶

Why It Matters¶

Not all errors are created equal, and error evolution reveals system dynamics. Fast-growing errors indicate sensitive dependencies or chaotic regimes. Saturating errors suggest bounded uncertainty. Correlated errors reveal shared root causes. Error tracking enables intelligent resource allocation: fix the fastest-growing errors first, accept errors that saturate quickly, and decompose correlated errors to address root causes.

The Key Move¶

For any forecast or estimate, explicitly track error as a function of time or iteration. Decompose total error into sources (initial condition error, model error, parameter error). Measure growth rates: linear, exponential, or saturating? Identify the timescale where error saturates (maximum uncertainty reached). Prioritize effort on errors that grow fastest or matter most for your decision. Update your confidence estimates as error grows.

Classic Application¶

Ensemble spread in weather forecasting. Initial condition errors grow exponentially in chaotic regimes (doubling every 1-2 days), roughly linearly in stable flows, and saturate when ensemble members span the full climatological range. Tracking spread growth tells forecasters when confidence collapses. If spread grows faster than climatology, low predictability; if slower, high predictability. This information is encoded directly in the forecast output.

Surprising Application¶

Project estimation in software development. Early estimates have large uncertainty from requirement ambiguity (fast-growing error source). As requirements solidify, that error saturates, but implementation uncertainty grows (new fast-growing source). Track which error dominates when. In early stages, invest in requirement clarification; in late stages, focus on technical risk. Error tracking reveals that a "95% confident estimate" in week 1 is meaningless - requirement error hasn't saturated yet - while the same confidence in week 8 might be reasonable.

Failure Modes¶

Assuming constant error growth - most systems have regime-dependent growth rates. Ignoring saturation - errors often hit natural bounds rather than growing indefinitely. Treating error components as independent - correlated errors invalidate simple decomposition. Over-focusing on total error - the error breakdown often matters more than the magnitude. Mistaking precision for accuracy - tracking error carefully doesn't reduce it unless you act on the information.

Go Deeper¶

Tribbia & Baumhefner, "The Reliability of Improvements in Deterministic Short-Range Forecasts in the Presence of Initial State and Modeling Deficiencies," Monthly Weather Review (1988); Palmer et al., "Representing Model Uncertainty in Weather and Climate Prediction," Annual Review of Earth and Planetary Sciences (2005).