The best content brief is the question nobody could answer

Help Center

Ask a team how they decide what documentation to write next and you will usually hear some combination of: whatever support complained about loudest this month, whatever the last big customer asked for, and whatever the person writing it happens to find interesting.

Meanwhile, if you run an AI assistant on your help center, there is a perfect list sitting right there. Every time someone asked a question your articles could not answer, that was a content brief, written by the person who needed it, in their own words.

We just never showed it to you properly. Now we do.

Conversations

The AI Chat page now has a Conversations tab: every chat a visitor had with your assistant, kept for 90 days.

Search works across message bodies, not just titles — searching refund finds conversations where somebody said "refund" even if the auto-generated title is about something else entirely. Each row shows how many questions were asked, how many answers cited an article, and whether any tools ran.

The filter that matters is No article cited. Those are the conversations in which no answer cited a help article — the assistant found nothing, or never looked. That is your writing queue, ranked by the only metric that counts: somebody actually wanted to know.

Open one and you see the transcript, which tools ran with which arguments, and — this turns out to matter more than we expected — the page the visitor was on when they asked. A question that reads as baffling in isolation is often obvious once you know they were staring at your pricing table when they typed it.

A number that read 157%

While building this we noticed the "Couldn't Answer" tile on the same page reporting 157.1%.

The cause was an honest mistake, and a common one. The numerator counted turns: every time the assistant answered without citing anything, add one. The denominator counted conversations: a set of unique session ids.

Those are not the same unit. One frustrated visitor asking eleven unanswered questions in a single session counts as eleven misses and one conversation. Divide, and you get 1100%.

Both sides now count the same thing — assistant turns that actually searched the help center. Which surfaced a second, subtler bug.

Not every unanswered question is a failure

The old counter incremented whenever an answer cited no article. But our system prompt explicitly tells the assistant to answer some things without searching: greetings, "what can you do?", clarifying questions back to the visitor. Those are correct behaviour. Searching your help center for "hi" would be the bug.

They were all being logged as content gaps.

So the fix has two halves: match the units, and only score turns where the assistant genuinely went looking. A greeting is no longer a documentation failure. The Content Gaps list is now questions that actually deserved an answer and did not get one.

The lesson we would generalise: when a percentage is wrong, the interesting bug is rarely the arithmetic. It is that the two numbers were never measuring the same thing, and nobody noticed until one of them got big enough to break the display.

The loop

It is short enough to do over a coffee:

  1. Filter by No article cited.

  2. Read a handful. Look for the same question asked three different ways.

  3. Write the article that should have existed. Publish it.

  4. The assistant picks it up immediately — no reindex, no rebuild.

Then check back in a fortnight and see whether that question stopped appearing. It is the only documentation feedback loop we know of that closes on its own.

What we store, and what we do not

Since this is a support inbox by another name, the boundaries matter.

Stored: the visitor's messages, the assistant's replies, which tools ran and their arguments, which articles were cited, and the URL and title of the page each question came from. Deleted automatically after 90 days.

Not stored: the page text the assistant read, anything passed as app context, and the results your own tools returned. Those are used to answer the question in front of them and discarded.

Visitors are anonymous unless your site identifies them. A conversation is tied to a browser session — the widget never asks for a name or an email, but your page can label it with identify(). But people type personal things into chat boxes, so any project member can read transcripts, and only owners and admins can delete them.


Live now on the AI Chat page. If you have had the widget running for a while, there is already a backlog waiting for you — and it is probably more honest about your documentation than your last content audit was.