When a retrieval system gives someone a wrong answer, the thing that makes it expensive is not the wrongness. It is that nobody knew.
The answer looked fine. It was fluent, confident, and formatted like every correct answer the system had produced that week. It sat in a support conversation for two days before somebody noticed the refund window it quoted did not exist. There is no exception in a log for that, no error rate that moved, nothing to alert on — which is why “we will catch it in review” turns out to mean “a customer will catch it”.
Grounding is the property that makes this measurable, and it is worth understanding precisely because the measurement is cheaper than almost anyone expects.
What grounding means
An answer is grounded when the claims in it are supported by the material the retrieval step actually pulled back. Not supported by the world in general, and not supported by the model’s training — supported by the specific passages that were in front of the model when it wrote that sentence.
That distinction is the useful part. It reframes hallucination from a vague failure of the model into a concrete relationship between two pieces of text you already have: the retrieved context, and the generated answer. Both are sitting right there in your request log. Comparing them is a mechanical operation.
The comparison is usually done with embeddings — turn the answer into a vector, turn the retrieved context into a vector, and measure how close they are. A small local embedding model does this perfectly well, with no external API call and no per-request cost, which matters when you intend to run it on every response rather than on a sample.
The result is a number between roughly zero and one. An answer that restates its source scores high. An answer that invents a different number, or drops a condition the source insisted on, scores noticeably lower — without any human having read either one.
What the score cannot tell you
This is where it is worth being honest, because a grounding score is often oversold.
It does not tell you the answer is correct. An answer can be beautifully grounded in a retrieved passage that was itself the wrong passage. Grounding measures faithfulness to the context, not truth.
It does not tell you which failure happened. The two causes of a bad retrieval answer are that the right passage never reached the model, or that it did and the model talked over it. A low grounding score is consistent with both. You still have to look at what was retrieved to know which one you are dealing with.
And it does not have a universal threshold. Where you draw the line between “fine” and “flag this” depends on your content, your embedding model and how much review capacity you have. Somewhere around the middle of the range is a common starting point, and the only way to set it properly is to run it over a few hundred of your own real answers and look at what falls either side.
What it does tell you is that something happened, automatically, on every single response, without waiting for a person to read them. That is a much lower bar than “detects hallucinations” and a much more useful one than nothing.
Where it belongs in a running system
Two places, and they reinforce each other.
Online, as a monitor. Score every response as it goes out. Log anything below your threshold with the query and the retrieved passages attached, so the review queue arrives pre-assembled rather than requiring somebody to reconstruct what happened. You are not blocking the response — you are making sure the bad ones are findable.
Offline, as part of your evaluation set. The same scoring function becomes the way you tell whether a prompt or retrieval change actually helped. Run it across a fixed set of cases before and after any change, and a tweak that quietly reduces grounding across the board shows up as a number moving rather than as a slow trickle of complaints three weeks later.
Using one function for both is deliberate. If the online monitor and the offline evaluation measure different things, they will eventually disagree, and you will not know which to believe.
Why this is worth building before you need it
The instrumentation argument for retrieval systems is always the same one, and it is always made too late. The information you need to diagnose a bad answer — the query, the retrieved passages, the generated response, and a score relating the last two — has to be captured at the moment the answer is produced. None of it can be recovered afterwards. A system that logs only the final answer is a system whose failures are permanently undiagnosable.
Adding a grounding score is one of the smaller pieces of that work, and it is the one that converts a category of failure from invisible to countable. Once it is a number on a graph, it becomes something a team can hold steady on purpose, and the safety layers around it have something concrete to act on.
Which is also why it comes up in interviews for anything involving retrieval. Describing the concept is easy. Being able to say where the score is computed, what it misses, and how you chose the threshold is the part that indicates you have run one.