Almost every AI support chatbot shows a row of sources under its answers. It is the feature that makes the answer feel accountable: here is where I got this.
In most implementations, that row is lying to you. Not maliciously — but it is answering a different question than the one the user is asking.
Retrieved is not the same as used
A retrieval-augmented chatbot answers roughly like this: search the knowledge base, put the top results into the model's context, and let it write. The obvious way to build the sources row is to show whatever came back from that search.
That is what most products do, and it is wrong in a specific way. A semantic search returns ten candidates because ten is a reasonable number to give a language model to choose from. The model then reads them and uses one. Showing all ten as sources tells the user that ten articles informed the answer, when nine were rejected.
The failure is quiet. Nobody files a bug that says "the sources row has too many entries in it." Users just learn, slowly, that the row is decorative — and stop reading it. You have paid the full engineering cost of citations and received none of the trust they were supposed to buy.
The list-everything problem
The failure gets more obvious the moment a user asks something structural.
"What do you have articles about?"
A good assistant answers with a list of your articles. Now the sources row contains every article in your help center, restating the exact links already in the answer directly beneath the answer. The user is looking at the same thirty titles twice.
That is the point where the abstraction visibly breaks, and it is worth taking seriously, because it reveals the actual bug: you cannot derive provenance from retrieval. You have to derive it from what the model did.
Three sets, not one
The fix is to stop treating this as one list. Any given answer has three distinct sets of documents behind it:
Retrieved — everything any tool returned during the turn. This is the set most products display. Its real job is different and important: it is the allow-list. If the model produces a citation to a document that was never retrieved, it invented the reference, and you drop it. That check is not theoretical. In our own testing, a model transcribing thirty document IDs got one wrong by a single character — the guard caught it and refused to turn a corrupted ID into a source.
Cited — the documents the answer actually links to. This is what makes inline citations clickable, and it is the honest basis for usage-based billing.
Consulted — the documents whose text the model read. This is the only one that belongs in a sources row.
The third set is the one nobody computes, and computing it is not hard. You already know which tool returned each document. Some tools hand the model real content: a search returning excerpts, or a "fetch this document" call returning a full body. Others hand it a table of contents: list the categories, list the articles, browse a section. Those return titles.
So:
The model opened a document in full → it is a source, whether or not the answer links it. Opening it is the provenance. If the model summarises an article and forgets to write a markdown link, that is a formatting lapse, not an absence of evidence.
A search returned it and the answer cited it → it is a source. The citation identifies which of the ten candidates was used.
The model only saw it as a title in a listing → it is never a source, even if the answer links it.
Under that rule, "what articles do you have?" produces a clean list of working links and no sources row at all, because the model read nothing. And "how do I configure SSO?" produces a paragraph with one or two chips under it — the articles it genuinely read.
The design detail that bites
There is a trap in implementing this, and it is worth naming because it is easy to get wrong in a way that looks fine until a user clicks something.
Your sources array usually does double duty. It is the data behind the chips, and it is the lookup table that makes inline citations clickable — the click handler finds the citation's document ID in that array and opens it.
So you cannot simply shrink the array to the consulted set. Do that, and every link in a listing answer goes dead: rendered as a citation, styled as a citation, and inert on click.
Keep the full cited set in the payload, mark each entry with a flag, and filter only at the point of display. One boolean per source, filtered in the renderer. The links keep working; the chips tell the truth.
Make the flag optional
If you ship this as an API change, make the field optional rather than required, and treat a missing value as "yes, show it."
That one decision means an old client against a new server keeps working, and a new client against an old server degrades to the previous behaviour instead of hiding every source. You will not deploy both sides at the same instant, and users have cached copies of your widget for longer than you would like.
Why it is worth the effort
A sources row is a claim: this answer came from these documents. If the claim is generated from retrieval, it is approximately true and users eventually notice. If it is generated from what the model actually read, it is precisely true, and it becomes the fastest way for your support team to audit an answer that went wrong.
There is a second payoff. Once "no sources" reliably means "the model read nothing," it becomes a real signal. An answer with no sources on a substantive product question is a documentation gap, and you can filter for those and turn them into a writing queue.
That only works if the row means something.
Kookee's AI chat assistant follows the rule above: the Sources row shows the articles an answer was built on, while every article the answer links to stays clickable. See How the assistant picks its sources for the specifics, or How to Build a Help Center for Your SaaS for the wider picture.