GLOSSARY
Ensemble
Combining multiple models' outputs for a better answer than any one alone — majority votes, best-of-N selection, or mixture-of-expert routing at scale.
One model errs idiosyncratically; several models err differently, so their agreement is signal and their disagreement is a flag. Ensembles range from simple (majority vote across outputs, best-of-N with a verifier) to architectural (mixture-of-experts routing each token to specialists) to product-level (Genspark-style multi-agent answers). The pattern underlies test-time-compute gains: spending extra inference to consult more opinions.
The costs are latency and spend multiplied by the number of opinions, and correlated errors — five models trained on similar data fail in similar ways, so diversity of training is what makes ensembles work. For high-stakes classification and research, ensembles are often the cheapest accuracy left to buy.