HALLUCINATIONS · CLAIM CHECKING
Debugging hallucinations: compare a claim with its source
Split an answer into claims, find the best supporting source sentence, and check numbers and citations before the answer ships.
Event trace
Computed data
Final metrics
Lexical checker over a fictional six-sentence source, not an entailment model or ADK feature. Support is the share of a claim's content words found in one source sentence; synonyms, negation and multi-sentence reasoning are not understood. Numbers are compared as text. No language model is called.
Treat each sentence as a claim
The checker splits the answer into sentences and compares each one with a fictional six-sentence source about the Ridgeline bridge. For every claim it finds the source sentence sharing the largest share of the claim's content words, then checks that every number in the claim appears in that sentence and that the claim's [S] citation points to it. The opening answer has three claims. C1 and C3 are supported by S1 and S4. C2 says the main span is 460 metres, but its best match S3 says 410.
Why a high overlap is not enough
C2 shares four of its five content words with S3, an overlap of 0.8, so a pure word-overlap check would accept it. The number check finds 460 missing from S3 and labels C2 contradicted. With When a claim fails set to block, the answer is withheld. Clear Every number must appear in the supporting sentence and the same answer passes with three supported claims: the wrong span length ships. Set the failure policy to deliver with flags and the answer reaches the user marked with the first problem, C2.
Unsupported and miscited claims
Load the invented-fact example. Its second claim, about a European engineering award in 2020, shares no content words with any sentence, so it is unsupported at any positive threshold. Load the wrong-citation example: the claim that Halvorsen Partners designed the bridge is true according to S5, but it cites S2, so it is labeled miscited. Clear Citation must point to the supporting sentence and the same answer is delivered with both claims supported. Correct facts with wrong citations still mislead a reader who checks the source.
Thresholds reject correct paraphrases
Load the correct paraphrase. Its second claim says the longest span measures 410 metres, while S3 says the main span is 410 metres long. Only three of five content words match, an overlap of 0.6. With the Support threshold at 0.5 it is supported; raise it to 0.65 and the true claim becomes unsupported and the answer is blocked. Edit the answer text directly to try your own claims; it is recomputed on every keystroke.
Limits of a lexical check
This model has no understanding of synonyms, negation, units or claims that need two sentences. Real grounding checks use retrieval plus an entailment or judge model, and those make their own mistakes. Use the lab to see why per-claim evidence, numeric checks and citation checks catch different failures. Play, Next event, Restart, Speed and Inspection time reveal the claims one by one over sixty seconds.