GROUNDING · SENTENCE BY SENTENCE
Output grounding check: inspect every factual sentence
Check every sentence of a cited answer against the passage it cites, and see which verifier catches a dropped condition, an unrelated source and an uncited claim.
Your experiment
A refund assistant answers in eight cited sentences. Four are supported; one drops the condition “only if defective”, one cites the warranty policy for a shipping claim, one has no citation and contradicts the policy, and one borrows a 30-day figure from the wrong passage. Find the verifier and failure action that keep the four good sentences and nothing else.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Fictional policy passages P1 to P5 and precomputed overlap scores; the claim check is modeled as correct on every claim except that it ignores conditions unless asked to check them, which a real entailment model will not match exactly. No model or retriever is called.
Eight sentences, four problems
A refund assistant’s answer has eight sentences with citations to policy passages P1 to P5. Four are supported. S2 drops a condition (P2 allows returns of opened electronics only if defective), S3 cites the warranty passage for a shipping-fee claim, S4 has no citation and contradicts P4, and S7 takes P1’s 30-day return window as a refund time. Verifier chooses what a sentence must show to pass, Overlap threshold applies to the overlap verifier, Claim check includes conditions extends the claim check, and On a failed sentence decides what the user then sees.
The opening run: overlap is not support
With lexical overlap at 0.4, S3 (0.12) and S4 (uncited) fail and are shown marked unverified, but S6, a correct paraphrase at 0.35, is flagged too. S2 copies its passage’s words (0.83) and S7 shares the 30-day phrase (0.45), so both pass and reach the user looking verified. Raise Overlap threshold to 0.5 and S7 fails, but S2 still passes; no threshold catches S2 before it rejects S1 (0.78), because the problem is a missing word, not missing overlap.
Verifiers compared
Set Verifier to A citation is present: only the uncited S4 fails, and three unsupported sentences reach the user unmarked. Set it to Claim check against the cited passage: S3, S4 and S7 fail and the paraphrase S6 passes, but S2 still passes, because the claim matches its passage if the qualifier is ignored. Tick Claim check includes conditions: S2 fails too, so exactly the four unsupported sentences fail.
Failure boundary: what happens to a failed sentence
With the full claim check, Show it marked unverified still puts four wrong statements in front of the user, with a label most readers skip. Remove the sentence delivers the four supported sentences and nothing else: GROUNDED. Replace the whole answer blocks it entirely, so the four correct sentences are lost as well. With the opening overlap verifier, Remove the sentence also deletes the correct paraphrase S6 and still shows S2 and S7: removing is only as good as the verifier that decides what to remove.
Reading the run and its limits
Play reveals the verdicts two sentences at a time, then the delivered answer and the outcome over sixty inspection seconds; the table lists each sentence, its citation, overlap, the truth and the verdict. Next event, Restart, Speed and Inspection time control the replay, and any input change recomputes it. The passages and scores are fictional and the claim check is idealised; real entailment checkers make their own mistakes, so measure them on labelled sentences from your own domain.