As large language models continue to be widely developed, robust uncertainty quantification techniques will become crucial for their safe deployment in high-stakes scenarios. In this work, we explore how conformal prediction can be used to provide uncertainty quantification in language models for the specific task of multiple-choice question-answering. We find that the uncertainty estimates from conformal prediction are tightly correlated with prediction accuracy. This observation can be useful for downstream applications such as selective classification and filtering out low-quality predictions. We also investigate the exchangeability assumption required by conformal prediction to out-of-subject questions, which may be a more realistic scenario for many practical applications. Our work contributes towards more trustworthy and reliable usage of large language models in safety-critical situations, where robust guarantees of error rate are required.

翻译：随着大语言模型被广泛开发，在高风险场景中安全部署它们需要可靠的量化不确定性技术。本研究探索如何将保形预测应用于语言模型在多项选择题回答这一特定任务中的不确定性量化。我们发现保形预测提供的不确定性估计与预测精度高度相关。这一发现可用于下游应用，如选择性分类和过滤低质量预测。我们还研究了保形预测对跨主题问题所需的交换性假设的适用性，这可能是许多实际应用中更为真实的场景。我们的工作有助于在需要稳定误差率保证的安全关键场景中更可信、更可靠地使用大语言模型。