Chunking
By HubAI editorial team · Source checked: 24 October 2026What does it mean?
Splitting source material into pieces for processing or retrieval. The size, boundaries and overlap of those pieces affect what an AI search tool can find and explain.
Picture this
A handbook is split by section, keeping a heading with the paragraphs it describes. A badly placed split could separate an exception from the rule it qualifies.
Where you’ll come across it
Retrieval-augmented generation and processing long documents. Test real questions, including ones whose answers cross a page or section boundary.
What it doesn’t mean
Smaller chunks are not always better. Pieces that lose context can produce misleading answers; very large pieces may crowd out other useful evidence.
Test whether the evidence survives
A short exercise · use fictional or non-sensitive material.
- Write a short fictional policy with one rule and one exception. Ask a question that needs both.
- Try a second question whose answer is deliberately missing from the policy.
- Check whether the answer points to the right passage and admits when the policy cannot answer.
Does the cited passage support this exact answer, including its exception?
Did that make it clearer?
Likes are a private on-page choice, not a public rating. For a correction or complaint, send the pre-filled form to our editorial inbox. Nothing is sent until you submit it.Check the source
Written and explained by HubAI. The reference below lets you go deeper; it is not an endorsement of a particular product.
Technical reference: huggingface.co ↗