GLOSSARY
Overfitting
When a model memorizes its training examples instead of learning the pattern — perfect on the practice test, brittle on anything new.
Overfitting is the classic failure of machine learning: the optimizer finds shortcuts that fit the training set exactly — including its quirks and noise — rather than the underlying rule. A fine-tuned classifier at 100% on training data and 60% on real data hasn't learned; it has memorized. The standard defenses are held-out validation sets, early stopping, regularization and more diverse data.
The concept now explains failures far beyond training runs: a model overfit to benchmark data performs beautifully on MMLU and badly on your task; an agent overfit to its eval suite passes tests while missing the point. Whenever results look too good, the first question is “what did it memorize?” — and the second is “show me the data it never saw.”