Large language models often produce confident but incorrect outputs, creating a critical need for reliable uncertainty quantification with formal abstention guarantees. We introduce information-lift certificates that compare model probabilities to a skeleton baseline, accumulating evidence through sub-gamma PAC-Bayes bounds that remain valid under heavy-tailed distributions where standard concentration inequalities fail. On eight diverse datasets, our method achieves 77.0\% coverage at 2\% risk, outperforming recent baselines by 10.0 percentage points on average. In high-stakes scenarios, we block 96\% of critical errors compared to 18-31\% for entropy-based methods. While our frequency-based certification does not guarantee severity-weighted safety and depends on skeleton quality, performance degrades gracefully under distributional shifts, making the approach practical for real-world deployment.
翻译:大型语言模型常生成自信但错误的输出,亟需具备正式弃权保证的可靠不确定性量化方法。本文提出信息提升认证机制,通过将模型概率与骨架基线进行比较,利用亚伽马PAC-Bayes边界累积证据,该边界在标准集中不等式失效的重尾分布下仍保持有效性。在八个多样化数据集上的实验表明,本方法在2%风险水平下达到77.0%的覆盖率,较现有基线平均提升10.0个百分点。在高风险场景中,本方法能拦截96%的关键错误,而基于熵的方法仅能拦截18-31%。尽管基于频率的认证无法保证严重性加权安全性且依赖骨架质量,但该方法在分布偏移下性能衰减平缓,具备实际部署的可行性。