GLOSSARY
Transformer
The neural-network architecture behind virtually every modern AI model — from chatbots to image generators — built around a mechanism called attention.
Introduced in the 2017 paper “Attention Is All You Need”, the transformer replaced step-by-step recurrence with attention: every part of the input can directly influence every other part, so the model captures long-range relationships and trains in parallel across thousands of GPUs. That parallelism is what made scaling to internet-sized datasets possible at all.
Everything in this directory traces back to it: GPT-style language models are transformer decoders, image generators like Stable Diffusion embed transformer blocks, and Whisper-style audio models are transformers too. When researchers talk about “scaling laws”, they mean how transformer quality predictably improves with more data, parameters and compute.