How to evaluate LLM hallucinations: a practical checklist

The word hallucination collapses several different defects, so one hallucination rate is not a reliable contract. Closed-book factuality asks whether a claim is correct against a verified external fact. Groundedness asks whether the supplied source supports the claim. Instruction faithfulness detects deviations from the input, while self-consistency detects contradictions within one answer. A response can be grounded in a wrong document or factually correct from model memory even though the supplied context does not support it.

Start with a decision contract: scenario, claim types, allowed sources, knowledge cutoff date, cost of error, correct abstention behavior, and the action after an uncertain result. NIST uses the more precise term confabulation for confidently presented false content and notes that the risk is especially important in long and domain-complex answers. The eval should name the concrete failure mode instead of attributing human intent to the model.

• Factuality → the claim matches an authoritative external fact at a fixed date.
• Groundedness → a specific evidence span supports the full scope of the claim.
• Consistency → the response does not contradict itself, the conversation history, or structured input.
• Calibration → confidence or the decision to answer matches the observed error frequency.

For short fact-seeking tasks, use questions with an unambiguous and stable answer plus an evidence record containing a URL or document ID, exact span, revision, verifiedAt, and reviewer. SimpleQA demonstrates a useful narrow design: a short answer and separate verdicts for correct, incorrect, and not attempted. But this benchmark does not establish quality for long-form answers, RAG, or your domain, so a public benchmark is a baseline rather than a release certificate.

Add your own slices: common fact, rare fact, ambiguous question, false premise, information after the cutoff, stale document, source conflict, no-answer, and questions with several valid forms. For grounded workflows include supportive, irrelevant, partially supportive, and contradictory context. Do not generate every question from the same chunks without review: synthetic sets often mirror source wording and make retrieval and grading unrealistically easy.

A single verdict for a paragraph hides partially correct answers. A claim extractor should identify externally verifiable statements, numbers, dates, entities, causal relations, and attribution while preserving scope and qualifiers. For each claim, the grader returns supported, contradicted, unverifiable, or not-in-context together with evidence IDs. Citation existence is not proof: the span must entail the actual claim, not merely discuss the same topic.

Measure claim coverage, factual precision, unsupported critical claims, and contradiction rate separately. The completeness extractor also needs calibration: if it misses an invented date, the downstream score becomes falsely high. Hand-label a representative slice, measure reviewer agreement, and analyze the disagreement taxonomy. A model grader is useful at scale, but its prompt, version, evidence order, and retry policy are part of the versioned eval rather than invisible infrastructure.

The eval must give the model a safe exit: ask a clarifying question, state that evidence is insufficient, or escalate to a human. Count correct, incorrect, and abstained as separate outcomes, then build a risk-coverage curve showing how error changes as the system answers a larger share of requests. High accuracy after refusing almost everything is not useful without coverage, while a high answer rate can hide dangerous guessing.

Do not rely only on confidence declared by the model. Calibrate it on held-out cases with reliability bins or a Brier score and compare it with simple signals such as evidence sufficiency, disagreement across samples, and verifier verdicts. Semantic entropy studies variation at the meaning level and can detect confabulations, but its authors distinguish these from systematic, consistent errors. An uncertainty detector therefore complements fact verification rather than replacing it.

Compare candidate and baseline on the same frozen cases, separately by language, domain, length, freshness, source, retrieval availability, and harm from error. An aggregate score may improve because of easy trivia while financial dates or medical qualifiers degrade. For high-risk slices, define critical invariants: no invented entity, number, or citation may reach the final result without verification or human review.

Test more than the model. Pin corpus, retriever, search API, prompt, citation resolver, claim extractor, verifier, and renderer versions. Inject empty retrieval, stale cache, broken source, partial document, conflicting revisions, and verifier timeout. If evidence is unavailable, the UI must not turn an unverifiable claim into a confident answer. Keep a redacted trace from input to rendered output so retrieval misses, generation defects, grader errors, and presentation bugs can be distinguished.

• Dataset health → label agreement, source freshness, leakage, and slice coverage.
• Answer quality → correct, incorrect, abstained, and risk-weighted critical failures.
• Evidence quality → claim coverage, citation entailment, and unsupported-claim rate.
• Operations → latency, grader disagreement, and cost per verified answer.

The decision record pins the dataset revision, sources, model, prompt, pipeline versions, primary metric, segment thresholds, critical failures, owner, and known-good rollback bundle. Promotion requires a practically meaningful paired non-regression, passing critical slices, and acceptable cost per verified answer. Do not claim a universal safe threshold or a model win from a vendor benchmark: your sources, languages, traffic mix, and consequences of error are different.

Rollout starts with offline replay, moves to shadow evaluation, a small canary, and then risk-bounded production. A confirmed user correction can become a regression fixture after privacy review; an implicit click or lack of complaint does not prove factuality. If a critical unsupported claim reaches a user, narrow coverage, disable the affected model or retrieval revision, restore the known-good bundle, and review already issued answers where the domain requires it. Production monitoring should use the same taxonomy as the offline eval so incidents do not disappear between differently named metrics.