In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential as factual knowledge bases; however, their capability to generate probabilistic knowledge about real-world events remains understudied. We explore utilizing the probabilistic knowledge inherent in LLMs to derive probability estimates for statements regarding events and their relationships within a BN. Using LLMs in this context allows for the parameterization of BNs, enabling probabilistic modeling within specific domains. Our experiments on eighty publicly available Bayesian Networks, from healthcare to finance, demonstrate that querying LLMs about the conditional probabilities of events provides meaningful results when compared to baselines, including random and uniform distributions, as well as approaches based on next-token generation probabilities. We explore how these LLM-derived distributions can serve as expert priors to refine distributions extracted from data, especially when data is scarce. Overall, this work introduces a promising strategy for automatically constructing Bayesian Networks by combining probabilistic knowledge extracted from LLMs with real-world data. Additionally, we establish the first comprehensive baseline for assessing LLM performance in extracting probabilistic knowledge.

翻译：本研究评估了大型语言模型通过近似领域专家先验知识来构建贝叶斯网络的潜力。大型语言模型已展现出作为事实知识库的潜力，但其生成现实世界事件概率知识的能力尚未得到充分研究。我们探索利用LLMs内在的概率知识，为贝叶斯网络中事件及其关系的陈述推导概率估计。在此背景下使用LLMs可以实现贝叶斯网络的参数化，支持特定领域的概率建模。我们在从医疗到金融领域的八十个公开贝叶斯网络上进行实验，结果表明：相较于随机分布、均匀分布以及基于下一词元生成概率的基线方法，通过LLMs查询事件条件概率能够获得有意义的结果。我们进一步探讨了这些LLM导出的分布如何作为专家先验知识来优化从数据中提取的分布，特别是在数据稀缺的情况下。总体而言，本研究提出了一种通过结合从LLMs提取的概率知识与真实世界数据来自动构建贝叶斯网络的前瞻性策略。此外，我们建立了首个评估LLMs在概率知识提取性能方面的综合性基线。