Zero-Hallucination RAG: The Grounding Contract Pattern

Most teams can build a RAG system that works in the demo. The hard part ships later: the day it answers a question the documentation never covered — confidently, fluently, and wrong. A retrieval pipeline does not prevent hallucination on its own. It only changes where the model gets its facts. Whether it stays inside those facts is a separate decision, and it has to be made in the architecture, not hoped for in the prompt.

On the streamerOS AI Support Agent I treat this as a hard contract I call grounding: the model may answer only from retrieved context, and when the context is empty it must refuse. This post is the pattern that enforces it.

Why "don't hallucinate" is not a strategy

A base model is a fluent prior over everything it ever read. Ask it about your product's config flags and it will happily interpolate plausible-sounding ones. Retrieval narrows the input, but if you paste the chunks into the prompt and ask politely for an answer, the model still treats its parametric memory as a fallback. The fix is to make the retrieved context the only admissible source and to define, explicitly, what happens when retrieval comes up empty.

The three boundaries

A grounding contract is three boundaries working together. Drop any one and the guarantee leaks.

1. Retrieval boundary

Embed the query, pull the top-k most similar chunks, and — critically — apply a similarity floor. A match below the floor is not a weak answer; it is no answer. This is what lets the system distinguish "the docs are thin here" from "the docs are silent here."

2. Prompt boundary

The system prompt names the retrieved context as the sole authority and spells out the refusal behaviour as a literal string. Vague instructions produce vague compliance; an exact refusal sentence produces an exact refusal.

3. Output boundary

Require attribution. If every claim must cite a source heading from the context, a fabricated claim has nowhere to anchor — the requirement makes invention structurally awkward rather than merely discouraged.

// retrieval boundary: a similarity floor turns "thin" into "silent"
const hits = await index.query({ vector: qVec, topK: 5, includeMetadata: true });
const ctx = hits.filter((h) => h.score >= 0.75);
if (ctx.length === 0) return REFUSAL;  // never reach the model

// prompt boundary: the model speaks only from ctx, and refuses otherwise
const answer = await generate({
  system: GROUNDING_CONTRACT,
  context: ctx.map((c) => c.metadata.text),
  query,
});

The refusal is a feature, not a failure

Engineering managers tend to flinch at "the bot says it doesn't know." They should celebrate it. A support assistant that cleanly defers the 5% it cannot answer is infinitely more deployable than one that invents an answer to all 100%. The refusal is also a free product signal: every clean "that isn't in my knowledge base yet" is a documentation gap, logged and rankable.

Hallucination is not a model defect you patch. It is an architectural gap you close — by deciding, in code, what the model is allowed to know.

The grounding contract is the spine of the streamerOS Support Agent. See the full retrieval and constraint breakdown in the project teardown, or the companion piece on running the whole pipeline at the edge.