GLOSSARY
Chunking
Cutting documents into retrieval-sized pieces before embedding — the unglamorous RAG decision that quietly decides answer quality.
A 200-page PDF cannot go into a context window whole, so RAG systems slice it into chunks and embed each one. Every knob matters: chunks too small lose the context that gives a sentence meaning; too large and retrieval drowns the answer in neighbors. Smart chunking respects structure — headings, paragraphs, tables — rather than blindly cutting every N tokens.
Teams debugging “RAG returns irrelevant results” usually find chunking before they find embeddings. It is the least glamorous lever in retrieval, and one of the highest-yield: the difference between a chunker that respects document structure and one that doesn't is often the difference between a demo and a product.