Courses / Agentic AI from Scratch (Python)

Session 16 of 40

Citing sources, saying "I don't know", and checking an answer is grounded

Notes

How to Make a RAG Chatbot Cite Sources, Say “I Don’t Know” and Check Groundedness

Agentic AI Foundations · Session 16 — Citing sources, saying “I don’t know”, and checking an answer is grounded

Before you start: what do you need to add citations and a groundedness check to RAG?

You need the code folder for this session open in your terminal. It holds helpdesk_rag.py (session 15’s helper with a quote field), check_grounded.py, run_test_set.py, build_index.py and load_policies.py (carried over from sessions 14 and 12), two prompt files in model_prompts/ (rag.md and grounded.md), data/test_set.json with thirteen cases, data/policies/ with the 5 policies, and two prompt files in prompts/. Install the libraries with pip install langchain langchain-openai langchain-chroma langchain-community langchain-text-splitters. You need the settings from session 1 (LLM_BASE_URL, LLM_API_KEY, LLM_MODEL) and EMBED_MODEL from session 13, which must be the model that builds the index. The folder policy_index/ is not in the lesson files: you build it with python build_index.py. Recap of session 15: the helper searches the index, puts the TOP = 4 closest chunks in the prompt and has the model return a typed reply with a source. Have paper or a notes app ready for the Flashcards and the Model answer.

Summary: how do you make a RAG answer cite its source, admit it does not know and stay grounded?

An answer from retrieved documents is useful when it can be checked. In this session the helper returns a quote, the exact words it relied on, next to the source file, and the code looks for those words inside a retrieved chunk. When no chunk supports an answer, the right reply is “I don’t know” with no citation and needs_human set to true, which the prompt permits and the code enforces. Groundedness goes one step further: it asks whether each claim in the answer is stated in the chunks, which a quote alone does not prove, so check_grounded.py asks a second model call to judge it. On our run the judge scored 4 of 4 answers as grounded and the test set passed 13 of 13.

Key concepts: quote, citation check, “I don’t know”, groundedness and judge defined

Term Definition
Quote The exact words of one clause that the model relied on, copied letter for letter, returned in the reply’s quote field.
quote_is_real A function that returns true only when the cited file is one of the retrieved chunks’ sources and the quote is inside that chunk, after squashing spaces and line breaks.
“I don’t know” reply An answer that starts with those words, names what is missing, cites nothing (section, source and quote are none) and sets needs_human to true.
Grounded Describes an answer whose claims are all stated in the chunks the model saw.
Ungrounded claim A statement in an answer that the chunks do not support, even if it is true in the real world.
Faithfulness A score from 0 to 1 from the Ragas library: the number of an answer’s claims supported by the retrieved context, divided by the total number of claims. Theory here: the code does not compute it.
Judge model A second model call that reads the chunks and an answer and returns a typed Verdict: grounded and unsupported.
context The text of the chunks the model saw, returned by ask_helper so the judge can read it.

Explanations: how do you make RAG answers checkable?

Why is a file name not enough as a citation?

Picture a student who writes footnotes from memory: each looks official, and one points to a page that does not exist. Session 15’s reply named a source file, but that is only something the model says. If the model names the wrong file, or a file it only skimmed, nothing in the reply gives it away. Anthropic’s guidance on reducing hallucinations makes the same point about auditability: make the response auditable by having the model cite quotes and sources for each of its claims.

A second picture is a lawyer quoting a clause: the quote and the contract go together, and anyone can open the contract and find the words. That is what the helper now does with the quote field.

How does the quote check work in code?

A footnote ties a claim to a page, and a quote ties it to a line you can search for. The reply class gained a fifth field:

class HelpDeskAnswer(BaseModel):
    answer: str = Field(description=...)
    section: str = Field(description=...)
    source: str = Field(description=...)
    quote: str = Field(description=...)
    needs_human: bool = Field(description=...)

model_prompts/rag.md asks for the exact words of ONE clause as the quote, without its number and without joining two clauses. Then quote_is_real in helpdesk_rag.py tests the quote in code:

def quote_is_real(reply, chunks):
    quote = " ".join(reply["quote"].split()).lower()
    for chunk in chunks:
        text = " ".join(chunk.page_content.split()).lower()
        if (chunk.metadata["source"] == reply["source"]
                and quote in text):
            return True
    return False

Two tests happen at once: the cited file must be the source of a retrieved chunk, and the quote must sit inside that chunk. Spaces and line breaks are squashed so a break in the middle of a clause cannot fail the match. In ask_helper, a reply that claims an answer (needs_human false) but fails this check is thrown away and replaced by an “I don’t know” answer with section, source and quote set to none. The prompt asks for a quote; the code decides whether to believe it.

Run python build_index.py (it prints Saved 34 chunks to policy_index/), then python helpdesk_rag.py. Ours printed:

Question: How much is my yearly learning budget?
retrieved:
  - learning-policy.md / 2. Learning budget
  - learning-policy.md / 3. How to request it
  - learning-policy.md / 4. Learning days
  - learning-policy.md / 1. Purpose
answer: Your yearly learning budget is 600.
section: learning-policy.md 2.1
source: learning-policy.md
quote: Every employee has a learning budget of 600 for each leave year.
needs_human: False

Open data/policies/learning-policy.md and find the quote word for word. That is what the check did.

How does the helper say “I don’t know” when the policies are silent?

A good colleague who is asked something outside their files says “I do not have that”, not a guess. Anthropic’s guide lists, among its basic techniques against hallucination, explicitly giving the model permission to admit uncertainty, and says this can drastically reduce false information. Two things make it work in the helper. First, rag.md says: if no piece supports an answer, do not guess; start the answer with “I don’t know”, say in a few words what is missing, set needs_human to true and set section, source and quote to none. It names three cases: the pieces do not answer the question, the person asks what a named policy says and no piece comes from a policy with that name, and the question is not about help desk topics. Second, ask_helper makes any answer that starts with “I don’t know” cite nothing and need a person, because a model can name a file it only skimmed.

See it fail the lazy way first. Ask about a policy that is not there: python helpdesk_rag.py "What does the travel policy say about booking flights?". The search still returns four chunks, because it returns its closest four whatever they are, and the expenses policy mentions travel. A careless helper would answer from them. Ours printed:

Question: What does the travel policy say about booking flights?
retrieved:
  - expenses-policy.md / 2. What you can claim
  - expenses-policy.md / 4. What you cannot claim
  - expenses-policy.md / 1. Purpose
  - learning-policy.md / 2. Learning budget
answer: I don't know what the travel policy says about booking flights, as there is no 'travel policy' provided in the given information.
section: none
source: none
quote: none
needs_human: True

The cause of a good reply here is the rule about named policies in rag.md: the expenses policy mentions travel, but it is not a “travel policy”. Saying “I don’t know” is a correct answer, not a failure. The gym question from the same README gives the same shape of reply: answer starting “I don’t know”, source and quote both none, needs_human true.

What is groundedness and how does the judge check it?

A newspaper fact-checker checks each claim against the file, not only the headline. A student who shows the page for each line of an essay does the same. An answer is grounded when each claim in it is stated in the chunks the helper saw; a claim that is true in the real world but absent from the chunks is not grounded. The Ragas documentation defines a faithfulness metric in the same spirit: it measures how factually consistent a response is with the retrieved context, ranges from 0 to 1, and is the number of supported claims divided by the total. Three of four supported is 0.75. check_grounded.py does not compute that number; it asks for a yes or no and the unsupported claims.

A cited answer can still say more than its quote. Suppose a reply quotes the real learning-budget clause but adds a deadline the clause does not state. The quote check passes, because the quote is real; the added claim is unsupported. The cause is that a quote checks where an answer points, not everything it says. So check_grounded.py adds a judge:

class Verdict(BaseModel):
    grounded: bool = Field(description=...)
    unsupported: str = Field(description=...)

judge = judge_prompt | model.with_structured_output(Verdict)
verdict = judge.invoke({"context": reply["context"],
                        "answer": reply["answer"]})

ask_helper now also returns context, the text the model saw, so the judge can read the same chunks. model_prompts/grounded.md treats an answer that starts with “I don’t know” as grounded, because it makes no claim. Each question costs two chat calls and one embedding call. Run python check_grounded.py. Ours printed, trimmed to two of the four questions:

Question : How much is the home-office allowance?
Answer   : The company pays a one-time home-office allowance of 150.
Source   : remote-work-policy.md | needs_human: False
Grounded : True | unsupported: none

Question : Does the company pay for my gym membership?
Answer   : I don't know if the company pays for gym memberships, as this is not mentioned in the provided policies.
Source   : none | needs_human: True
Grounded : True | unsupported: none

Grounded 4 of 4

A judge is a model too, so it can be wrong; treat “Grounded” as one more signal. Anthropic’s guide itself adds that these techniques reduce hallucinations but do not eliminate them, and that critical information should still be validated.

Check How it works In today’s code?
Source check The cited file must be a retrieved one Yes, inside quote_is_real
Quote check The quote must be inside that chunk Yes, quote_is_real
Judge model A second call lists claims the chunks do not support Yes, check_grounded.py
Metric or managed service A faithfulness score, or a grounding check in a cloud service No: theory only

As an example of the last row, Amazon Bedrock Guardrails offers contextual grounding checks that detect and filter hallucinations when a reference source and a user query are supplied. They check grounding and relevance, produce confidence scores and let you set thresholds. AWS’s page says the Classic tier does not support conversational use cases, so read which tier a managed check uses before relying on it for a chat feature. This code does not use such a service.

How does the test set check “I don’t know”?

data/test_set.json has thirteen cases: session 15’s eleven, plus two questions the policies do not answer (gym membership and notice period). In run_test_set.py the cases without an answer expect needs_human true, the words “I don’t know” in the answer and a source of none. Run python run_test_set.py. Ours printed:

PASS - carry-over days -
...
PASS - no clause supports it -
PASS - notice period -

Passed 13 of 13

Run it more than once. One early run of ours failed two cases, and both failures are why the code looks as it does.

Three situations where the checks earned their place

  1. The model joined two clauses into one quote, and the check rejected a real answer. A joined text is inside no single chunk, so rag.md now asks for ONE clause.
  2. A case failed with source was 'x', expected 'none'. The model cited a file on an “I don’t know” answer, which the second block in ask_helper now clears.
  3. The judge says grounded: False for a good answer. The judge can be wrong: read unsupported, and either make grounded.md clearer or accept the miss.
What you learned The design decision Where it shows
A file name is only something the model says Ask for a quote and check it in code quote_is_real
Silence is a valid answer “I don’t know”, no citation, a person needed rag.md, ask_helper
A real quote does not prove every claim Add a judge that reads the chunks check_grounded.py
A judge can be wrong Treat its verdict as one signal Grounded X of Y

Basic assignment: in quote_is_real, make the check always return True. Ask a question the policies do not answer a few times. Does an invented citation get through? Then delete the “I don’t know” paragraph from model_prompts/rag.md and run the test set. Which cases break?

Project step: cite, refuse and check. Run python build_index.py, python helpdesk_rag.py "Does the company pay for my gym membership?", python check_grounded.py and python run_test_set.py in order and write down your grounded count and pass count. Add three cases to data/test_set.json: two answerable and one not. The build prompts are prompts/1-write-helpdesk-rag.md and prompts/2-write-check-grounded.md. Stop when both scripts run and you have two counts.

Practice: explain and diagnose. First, answer the Model answer question below aloud in three beats before reading it. Second, for each situation, say what the code should do:

  1. The model quotes two joined clauses and cites a real file.
  2. The model answers “I don’t know” but names leave-policy.md as the source.
  3. The quote is real, but the answer adds a deadline the clause does not state.

Flashcards: can you answer these citation and groundedness questions aloud?

Say the answer aloud before revealing it. Speaking is a different skill from reading, so practise both.

1. What is the quote field for? It holds the exact words of one clause the model relied on, so that code can search the retrieved chunks for them. A file name alone is something your code cannot check.
2. What two things does `quote_is_real` test? That the cited file is the source of a retrieved chunk, and that the quote sits inside that chunk once spaces and line breaks are squashed to single spaces and the case is lowered.
3. What happens when a claimed answer fails that check? `ask_helper` throws the answer away and replaces it with an "I don't know" reply, with `section`, `source` and `quote` set to `none` and `needs_human` true.
4. Why is "I don't know" a correct answer? When no chunk supports an answer, a confident reply would be invented. Admitting it is the accurate reply, and vendor guidance lists permission to admit uncertainty as a basic technique against hallucination.
5. Why does `ask_helper` clear the citation on an "I don't know" answer? A model can name a file it only skimmed. An answer that says it does not know should cite nothing, so the code sets `section`, `source` and `quote` to `none`.
6. What is groundedness? An answer is grounded when each claim in it is stated in the chunks the helper saw. It is about the claims, not only about the citation.
7. Why can a real quote sit beside an unsupported claim? A quote shows where an answer points, not everything it says. The answer can add a detail, such as a deadline, that the cited clause does not state.
8. What does the judge in `check_grounded.py` return? A typed `Verdict` with `grounded`, true or false, and `unsupported`, the claims no chunk supports or "none". It reads the `context` the helper saw and the answer.
9. Why treat the judge's verdict as one signal? A judge is a model too, so it can be wrong. If it flags a good answer, read `unsupported` and decide; if it passes a bad one, other checks still matter.
10. Why include questions the policies do not answer in the test set? They exercise the "I don't know" path. A helper tested only on covered questions looks better than it is, because it is never tested on admitting it does not know.

Model answer: how do you explain making an AI assistant’s answers trustworthy?

The core question this session answers: “A colleague asks: how do we know the helper’s answer is real, and what does it do when the policies are silent?”

Three beats, in this order: cite a quote the code checks · say “I don’t know” · check the answer is grounded. Skip any of these three and the answer is incomplete.

  1. Cite a quote the code checks: the reply carries a source and a quote, and quote_is_real looks for the quote inside a retrieved chunk of that file.
  2. Say “I don’t know”: when no chunk supports an answer, the answer says so, cites nothing and sets needs_human to true.
  3. Check the answer is grounded: check_grounded.py asks a judge model whether each claim is in the chunks, because a cited answer can still say more than its quote.

The full answer, said aloud:

(Cite a quote the code checks) “The helper returns a source file and the exact words it relied on. My code searches the retrieved chunks for those words, and an answer whose quote is not there is replaced by an I don’t know.

(Say I don’t know) If no chunk supports an answer, the helper says it does not know, cites nothing and flags that a person is needed. That is a correct answer, and I test it with questions the policies do not cover.

(Check the answer is grounded) A quote shows where an answer points, not everything it says, so a second model call checks each claim against the chunks. It is one more signal, because a judge can be wrong.“

Missing a beat is a failure state. The follow-up question lands where you went quiet: skip the last beat and the next question is how you know the claims are right. Practise until the three come out without prompting.

Why this matters: where do citations and groundedness checks matter at work?

In a compliance conversation: reviewers ask where an answer came from and what the system does when it does not know. A quote you can find word for word in a file, and an “I don’t know” reply that cites nothing, are answers you can show.

On the job: an assistant that says “I don’t know” when the policies are silent gives a support lead something to check, not a guess to second-guess. A grounded count and a pass count turn “seems fine” into two numbers you can compare between versions.

In the rest of this course: session 17 compares retrieval settings so the right chunks are found more often, and the evaluation sessions turn groundedness into a measured metric.

After this unit: what to build after your helper cites its sources

Assignment. Make the check always return True, ask a question the policies do not answer a few times, and say whether an invented citation got through. The brief for the project work, cite, refuse and check, is the project file.

What’s next. Session 17 covers better search: top-k, a score threshold, MMR and metadata filters, compared side by side on the same questions.

Want help building this with someone?

Tell us what you are working on and we will write back about a mentor-led cohort, live sessions or 1:1 training. It is only an enquiry: you choose whether to go further.

Ask about mentor-led training

TechEazy Consulting training material — not affiliated with or endorsed by OpenAI, Anthropic, LangChain or any other company named in this course.