Grounded answers: passage citations, refusals and support tests

Connecting a model to search does not, by itself, stop it from making things up. It changes what the model has in front of it, and it gives you a way to check the answer afterwards. Grounding is the discipline that turns those two facts into reliable behavior: retrieve the right passages, make the answer depend on them, show which passage supports which claim, refuse when nothing does, and test that the whole chain holds. This page walks through each step for developers adding retrieval to an agent or an AI feature.

What retrieval fixes and what it does not

A model without retrieval answers from patterns in its training data. When the question is about something recent, niche or internal, it fills the gap with text that sounds right. Retrieval helps in three specific ways: it supplies facts the model never saw, it supplies newer versions of facts the model saw in an older form, and it gives every claim something to point at.

It does not fix everything. Several common failures survive retrieval untouched:

• Ignoring the context. The model answers from memory even though the retrieved passage says something different.
• Over-synthesis. Two passages each support half a claim, and the model joins them into a whole claim neither source makes.
• Misattribution. The fact is real, but the citation points to a passage that does not contain it.
• Bad retrieval. The search returned an outdated, low-quality or off-topic page, and the model faithfully summarized it.
• Filling silence. Nothing relevant came back, and the model answered anyway.

Each of these needs its own countermeasure. The rest of this page takes them in pipeline order.

The grounding pipeline

A grounded answer passes through five stages. Keeping them separate in code makes each one testable on its own.

1. retrieve   query a search source -> candidate documents
2. select     split into passages, keep the ones that bear on the question
3. answer     generate only from the selected passages, each with an id
4. cite       attach passage ids to every factual sentence
5. verify     check each claim against its cited passage; drop or flag failures

Whether stage one is a live web search API, an index of your own documents or both depends on the question. Current events, prices and recent releases need fresh web results. Policies, contracts and internal decisions belong in your own document set, where the web is simply the wrong source. Many teams build both and route by question type; RAG with live web data covers the retrieval side in depth.

Feed passages, not pages

The unit you hand to the model matters. A whole page buries the relevant sentence among navigation, banners and unrelated sections, and it makes citations vague. Split documents into passages of a few sentences to a few paragraphs, give each a stable identifier, and keep the metadata a reviewer will need:

{
  "id": "p-0412",
  "source_url": "https://example.org/policy/returns",
  "title": "Returns policy",
  "retrieved_at": "ISO-8601 timestamp",
  "text": "Items may be returned within the period printed on the receipt..."
}

The retrieval date is not decoration. A passage that was true when fetched may be false now, and a reviewer can only judge that if the date travels with the claim. Two more rules pay off quickly. Put only passages that cleared a relevance threshold into the context, rather than everything the search returned. And decide in advance which domains or collections may answer at all: a short allowlist beats a long ranking. The note on deciding which sources may answer goes into that choice.

Cite at the passage level

A citation that points to a homepage or a fifty-page PDF is barely better than none, because the reader cannot check it in reasonable time. Ask for citations at the passage level, attached to the sentence they support, and have the model return structured output so your code can validate it.

{
  "answer": [
    {"claim": "Returns are accepted within the period printed on the receipt.",
     "support": ["p-0412"]},
    {"claim": "Opened software cannot be returned.",
     "support": ["p-0415"]}
  ],
  "unsupported": ["Whether gift cards can be refunded"]
}

Three checks can run on this output without any model at all. Does every cited id exist in the context you supplied? Does every factual claim carry at least one id? If you also asked for a short supporting quote, does that quote appear word for word in the passage? A citation that fails any of these is not evidence, however tidy it looks. Models can produce a plausible URL or a neat footnote for a source they never saw, so citations have to be checked against what was actually retrieved, not against what looks right. Citations that hold up goes further into what makes a reference reviewable.

Make refusal a normal output

One of the most useful changes you can make is giving the model a legitimate way to say that the sources do not answer the question. If the only accepted output is an answer, you will get answers.

In practice that means:

• A dedicated field, like unsupported above, for questions or sub-questions the passages do not cover, instead of an instruction buried somewhere in a long prompt.
• Partial answers: answer the supported half, list the unsupported half. Users accept this more readily than a confident whole that later turns out wrong.
• A distinction between “searched and found nothing” and “search was unavailable”. The first is an answer. The second is an error that should be retried or reported, never converted into “no information exists”.
• Interface copy that treats a refusal as a result. If the product makes an empty answer look like a failure, someone will eventually tune the prompt until refusals disappear.

The note a refusal is a result argues the product side of this.

Test claim support, not answer quality

Asking whether an answer looks good is the wrong test. Grounding is tested claim by claim: is each sentence supported by the passage it cites? Build a small evaluation set and run it on every prompt, model or retrieval change.

1. Collect questions from real usage, including ones whose correct answer is “not in the sources”.
2. Freeze the context for each question, so failures can be traced to generation rather than to shifting search results. Test retrieval separately.
3. Split answers into atomic claims, one fact per claim.
4. Label each claim against its cited passage: supported, contradicted, or not found in the passage. A second model can do first-pass labelling; a person should review a sample and every disagreement.
5. Track three counts over time: unsupported claims, wrong citations, and questions that were answered when the expected output was a refusal.

A sample test case looks like this:

Question: “Can opened software be returned?” Context: passages p-0412 to p-0418 from the returns policy. Expected: a claim that opened software cannot be returned, citing p-0415. Fail if the answer cites any other passage for that claim, adds a condition the passage does not state, or leaves the claim uncited.

Keep the refusal cases in the set permanently. They are the first to regress when someone tries to make the system more helpful.

Grounding in a governed workspace

GoCux applies the same principles to a team's own material rather than to the open web. It is not a web search API and it does not read the open web. Knowledge search answers a question scoped to one project's documents and returns passages with their citations, and an empty result counts as a valid answer rather than a failure. Web pages reach a project only through an authorized browsing or export step, and once imported they keep their original page reference and the date they were retrieved.

In research work, a finding backed by a quotation is stored apart from a question that no source could answer, so gaps stay visible instead of being smoothed over; missing evidence is an open question explains why that separation matters. A citation there still has to be read. It shows where to look, and the reviewer decides whether the passage really says what the claim says.

Checklist for a grounded feature

• Retrieval returns passages with stable ids, source URLs and retrieval dates.
• Only passages above a relevance bar, from allowed sources, reach the model.
• Every factual sentence carries at least one passage id, and code rejects ids that were not in the context.
• The output format has a place for unsupported questions, and the interface shows it plainly.
• “Nothing found” and “source unavailable” are different states in code and in the interface.
• An evaluation set with frozen contexts, refusal cases included, runs on every change.
• A person reviews a sample of labelled claims, not just the automated score.