Large language models primarily rely on inductive reasoning for decision making. This results in unreliable decisions when applied to real-world tasks that often present incomplete contexts and conditions. Thus, accurate probability estimation and appropriate interpretations are required to enhance decision-making reliability. In this paper, we propose a Bayesian inference framework called BIRD for large language models. BIRD provides controllable and interpretable probability estimation for model decisions, based on abductive factors, LLM entailment, as well as learnable deductive Bayesian modeling. Experiments show that BIRD produces probability estimations that align with human judgments over 65% of the time using open-sourced Llama models, outperforming the state-of-the-art GPT-4 by 35%. We also show that BIRD can be directly used for trustworthy decision making on many real-world applications.

翻译：大型语言模型主要依靠归纳推理进行决策，这使得它们在应用于常呈现不完整上下文和条件的现实任务时，会做出不可靠的决策。因此，需要准确的概率估计和恰当的解释来增强决策的可靠性。本文提出了一种名为BIRD的贝叶斯推理框架，用于大型语言模型。BIRD基于溯因因素、LLM蕴含以及可学习的演绎贝叶斯建模，为模型决策提供了可控且可解释的概率估计。实验表明，使用开源的Llama模型时，BIRD生成的概率估计与人类判断的一致性超过65%，优于最先进的GPT-4达35%。我们还证明了BIRD可直接用于许多现实应用中的可信决策制定。