Retrieval, source hierarchy and keeping context clean
Retrieval is how a model answers from documents it was never trained on. The quality of the answer is mostly the quality of what was retrieved.
What you will be able to do
Explain retrieval well enough to debug it, and set a source hierarchy that survives contradiction.
Before you start
- Comfort with the context window and context engineering guides.
How retrieval works
Documents are split into chunks and indexed. A query finds the chunks that look most relevant. Those chunks go into the context. The model answers from them.
Every failure mode lives in that pipeline: chunks split mid-argument, the retrieved passage is topically similar but not the answer, the index is stale, or the right document was never in it.
Debugging by stage
When a retrieval-backed answer is wrong, ask which stage failed. Was the document indexed? Was the right chunk retrieved? Was the chunk sufficient? Did the model use it or override it with its own priors?
Most systems can show you the retrieved passages. Look at them before blaming the model.
Hierarchy and freshness
When sources disagree, something has to win. State the order: the current official document beats a blog post; the signed contract beats the summary; the primary study beats the press release.
Attach dates. A model has no way to know that one of your documents was superseded unless you say so.
What good looks like
- You can inspect what was retrieved.
- Your sources carry dates and a stated precedence.
- The system says when the answer is not in the corpus rather than improvising.
What it cannot do
- Retrieval reduces fabrication; it does not remove it.
- Retrieval quality is bounded by the corpus. Missing documents produce confident gaps.
When it goes wrong
- Confident answer contradicting your documents.
- Check what was retrieved. The model may be answering from training rather than from context. Instruct it to answer only from the supplied sources and to say when it cannot.
- Right document, wrong passage.
- A chunking problem. Chunk on semantic boundaries rather than fixed character counts, and overlap them.
Privacy and data handling
An index is a copy. Whatever governance applies to the source documents applies to it, including deletion requests.
Sources3 referenced
- Claude developer documentation
Anthropic · official product documentation · link checked 2026-09-06
- OpenAI platform documentation
OpenAI · official product documentation · link checked 2026-09-06
- AI Risk Management Framework
NIST · standards framework · link checked 2026-09-06
Links are checked at build time. If one has moved, tell us.
Read next
What context engineering actually means
Prompting is what you ask. Context engineering is what you put in the room before you ask it. The second is where most of the quality lives.
Checked 2026-09-06
Research and learn with a verification trail
Used carelessly these tools produce confident summaries of nothing. Used with a trail, they are a genuine accelerator.
Checked 2026-09-06
Create a source-backed research workflow
A short researched piece where every retained claim traces to a source you personally opened.