Transfer Learning (Leveraging Related Experience)¶

Why It Matters¶

Learning from limited data is the norm, not the exception. Starting from scratch ignores all accumulated experience. Transfer learning formalizes: "How can I leverage related problems I've already solved?" This is how human learning actually works - we never start from zero.

The Key Move¶

Before collecting more data or training from scratch, ask: "What related problem has already been solved with abundant data?" Use pre-trained models or representations as starting points, then adapt to your specific case. The more similar the source and target tasks, the more transfer helps. Even distant transfer often beats random initialization.

Classic Application¶

Computer vision. Models pre-trained on ImageNet (millions of labeled images) transfer remarkably well to specialized tasks (medical imaging, satellite imagery, microscopy) with only hundreds of examples. The low-level features (edges, textures) and mid-level features (shapes, parts) generalize broadly.

Surprising Application¶

Career transitions and skill acquisition. When learning a new skill, identify related skills you've already mastered. A physicist learning machine learning can transfer mathematical intuition, debugging strategies, and research methodology. Don't start from "beginner" - explicitly map what transfers and build on it.

Failure Modes¶

Negative transfer - when source and target are superficially similar but structurally different, transferred knowledge hurts. Frozen representations - if you don't fine-tune, the model can't adapt to domain-specific patterns. Over-reliance - sometimes you have enough data to train from scratch, and transfer just adds bias.

Go Deeper¶

Pan & Yang (2010), "A Survey on Transfer Learning," IEEE TKDE; Yosinski et al. (2014), "How Transferable Are Features in Deep Neural Networks?" NIPS