Faithfulness Checker for LLMs
Verify the fidelity of an LLM output with respect to its source context RAG. Analyze sentence by sentence: supported, not supported, contradicted. Score from 0-100 with numerical contradiction detection. Client side only, no data sent.
Input field
Come utilizzare LLM Faithfulness Checker
Attach source and output
Incolla il contesto fornito al sistema RAG nella prima textarea e la risposta generata dall'LLM nella seconda (fino a 6000 caratteri di contesto, 2000 di risposta).
Start verification
The output is segmented into individual sentences; for each, the lexical overlap with the context and numerical value and date inconsistencies are detected.
Read scores and breakdown by sentence
Every sentence receives a verdict (supported, not supported, contradicted). The final score is: (supported x1.0 + not supported x0.3 - contradicted x0.5) / total x 100.
Suggerimenti
- Use complete source contexts and not truncated ones: partial contexts generate false "not supported" even for correct sentences.
- Be concise — keep similar length. Pay special attention to phrases marked with "contradicted": they are the strongest indicator of numerical data hallucination.
- Check the Best Practice Panel to compare the methodology of this tool with RAGAs, TruLens and Anthropic Citation Prompting.
Domande frequenti
How is loyalty score calculated?
Score is calculated by the formula (supported sentences x1.0 + unsupported sentences x0.3 - disputed sentences x0.5) divided by total sentences, multiplied by 100, within a range of 0-100.
What's the difference between "not supported" and "contradicted"?
Not supported means the phrase lacks sufficient lexical support in context but does not explicitly contradict it. Not contradicted indicates a detected conflict, typically on numeric or date values differing from those in the context.
Does the tool use an LLM for analysis?
No, analysis is completely heuristic, based on sentence segmentation and lexical overlap calculated client-side in JavaScript: no API calls, no data sent to external servers.
What are "unattached key terms"?
Words or entities relevant to the output but absent from the source context: they indicate potential additional details provided by the model without basis in the data.
Does this tool replace frameworks like RAGAs or TruLens?
No, it's a quick and free heuristic check for a single comparison context/output. For systematic evaluations on production test datasets, dedicated frameworks like RAGAs, TruLens or DeepEval are recommended, also mentioned in the best practice section of the tool.