As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for the specific task of multiple-choice question-answering. We find that the uncertainty estimates from conformal prediction are tightly correlated with prediction accuracy. This observation can be useful for downstream applications such as selective classification and filtering out low-quality predictions. We also investigate the exchangeability assumption required by conformal prediction to out-of-subject questions, which may be a more realistic scenario for many practical applications. Our work contributes towards more trustworthy and reliable usage of large language models in safety-critical situations, where robust guarantees of error rate are required.

翻译：随着大语言模型的广泛开发，在高风险场景中安全部署所需的鲁棒不确定性量化技术将变得至关重要。本研究探讨了如何利用共形预测为语言模型在多项选择题解答这一特定任务中提供不确定性量化。我们发现，共形预测生成的不确定性估计与预测准确率之间存在紧密相关性。这一发现可应用于选择性分类、过滤低质量预测等下游任务。我们还针对共形预测所需的可交换性假设在跨主题问题场景（实际应用中更常见的场景）中的有效性进行了研究。本研究为在需要严格错误率保障的安全关键场景中更可信、更可靠地使用大语言模型做出了贡献。