GLOSSARY
Distillation
Training a small, fast model to imitate a large one — capturing much of the big model's quality at a fraction of the cost to run.
Instead of learning only from human-written text, a distilled model learns from the outputs of a stronger teacher model — absorbing its judgment, style and edge cases. The student can then run many times faster and cheaper, which is how products put “mini” and “flash” tiers into your hands at low latency.
The catch is knowledge narrowing: distilled students are weaker at the long tail the teacher saw in training. Distillation also sits at the center of an industry debate — several providers' terms restrict training competitors on their outputs, which turned “distilling a frontier model” from an engineering phrase into a legal question.