The propensity of Large Language Models (LLMs) to generate hallucinations and non-factual content undermines their reliability in high-stakes domains, where rigorous control over Type I errors (the conditional probability of incorrectly classifying hallucinations as truthful content) is essential. Despite its importance, formal verification of LLM factuality with such guarantees remains largely unexplored. In this paper, we introduce FactTest, a novel framework that statistically assesses whether a LLM can confidently provide correct answers to given questions with high-probability correctness guarantees. We formulate factuality testing as hypothesis testing problem to enforce an upper bound of Type I errors at user-specified significance levels. Notably, we prove that our framework also ensures strong Type II error control under mild conditions and can be extended to maintain its effectiveness when covariate shifts exist. Our approach is distribution-free and works for any number of human-annotated samples. It is model-agnostic and applies to any black-box or white-box LM. Extensive experiments on question-answering (QA) and multiple-choice benchmarks demonstrate that FactTest effectively detects hallucinations and improves the model's ability to abstain from answering unknown questions, leading to an over 40% accuracy improvement.

翻译：大语言模型（LLM）倾向于产生幻觉与非事实内容，这削弱了其在关键领域应用的可靠性，而此类领域必须严格控制Ⅰ类错误（将幻觉错误分类为真实内容的概率）。尽管该问题至关重要，但具备此类保证的LLM事实性形式化验证研究仍处于空白。本文提出FactTest——一种新颖的统计框架，用于评估LLM能否以高概率正确性保证对给定问题提供可靠答案。我们将事实性检验构建为假设检验问题，从而将Ⅰ类错误上界控制在用户指定的显著性水平内。值得注意的是，我们证明了该框架在温和条件下同样能实现强Ⅱ类错误控制，并可扩展至存在协变量偏移的场景。本方法具有无分布特性，适用于任意数量的人工标注样本；同时具备模型无关性，可应用于任何黑盒或白盒语言模型。在问答（QA）与多项选择基准测试上的大量实验表明，FactTest能有效检测幻觉现象，并提升模型对未知问题的拒答能力，实现超过40%的准确率提升。