Skip to index

GLOSSARY

Token

The small chunks of text (roughly a word or part of a word) that AI models actually read and generate; pricing and context limits are both counted in tokens.

Models do not read letters or whole words; they read tokens — chunks of text a few characters long. English prose averages roughly 1.3 tokens per word, while other languages and code tokenize differently, which is why quality and cost vary across languages.

Tokens matter practically in two places: context windows (how many tokens a model can consider at once) and pricing (APIs bill per token processed). A '1M token context' can hold entire books; a per-token bill is why long conversations and large files cost more.

Related terms

Tools that use this

Related categories