Skip to content

Local vs Global Optima

Why It Matters

Gradient-following (always improving) gets stuck at local optima. Getting to global requires either accepting temporary worsening or starting from multiple points.

The Key Move

Ask: if I'm at what seems optimal, is it because there's nothing better anywhere, or just nothing better nearby? Methods to escape local optima: random restarts, simulated annealing (accepting worse moves with some probability), or restructuring the problem.

Classic Application

Machine learning. Gradient descent finds local minima. Deep learning works partly because the local minima are often good enough, and partly through tricks to escape bad ones.

Surprising Application

Career paths. Your current job may be locally optimal (can't improve with small changes) but globally suboptimal (a completely different field would be better). Escaping requires accepting a temporary step down.

Failure Modes

  • Assuming local optimum is global
  • Constantly abandoning local optima without reaching global (endless restarts)
  • Ignoring that the cost of transition may exceed the benefit of global optimum

Go Deeper

Optimization literature. Kauffman's "NK landscapes" for intuition on landscape ruggedness.