Feature Engineering vs. Feature Learning¶

Why It Matters¶

Most reasoning failures stem from measuring the wrong things. Traditional approaches assume someone knows what features matter. But often the most predictive patterns aren't the obvious ones, and manual feature design embeds your existing biases. Conversely, pure feature learning can discover spurious patterns that don't generalize.

The Key Move¶

Distinguish between the representation (what you measure) and the model (how you combine measurements). When predictions fail, ask: "Am I measuring the right things?" before asking "Am I combining them wrong?" Consider whether you need more domain insight (better feature engineering) or more data to discover patterns (feature learning).

Classic Application¶

Computer vision. Hand-engineered features (edges, corners, textures) dominated for decades. Deep learning's breakthrough was learning features directly from pixels - discovering that mid-level representations like textures and shapes emerge automatically from prediction pressure, often better than human-designed features.

Surprising Application¶

Self-knowledge and personal development. Asking "am I happy?" (hand-engineered feature) might miss what actually matters. Instead, track raw data (sleep, social interactions, flow states) and look for patterns in what predicts your wellbeing. You might discover that "time in nature" matters more than you'd have guessed by introspection alone.

Failure Modes¶

Feature learning requires massive data - you can't discover patterns you don't have examples of. Uninterpretable features - learned representations often defy human understanding, making debugging impossible. Spurious patterns - without domain constraints, models learn non-causal correlations.

Go Deeper¶

Bengio, Courville & Vincent (2013), "Representation Learning: A Review and New Perspectives," IEEE TPAMI; Domingos (2012), "A Few Useful Things to Know About Machine Learning," CACM