Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide them into a sparse collection of smaller ones, underlies Judea Pearl's causality, and determines their explainability and interpretability. Despite their popularity, there are almost no resources in the literature on how to compute Shannon's entropy and the Kullback-Leibler (KL) divergence for BNs under their most common distributional assumptions. In this paper, we provide computationally efficient algorithms for both by leveraging BNs' graphical structure, and we illustrate them with a complete set of numerical examples. In the process, we show it is possible to reduce the computational complexity of KL from cubic to quadratic for Gaussian BNs.

翻译：贝叶斯网络（BNs）是机器学习与因果推断中的基础模型。其图结构能够处理高维问题，将其分解为稀疏的小规模问题集合，支撑了朱迪亚·珀尔的因果理论，并决定了模型的可解释性与可解读性。尽管贝叶斯网络应用广泛，但现有文献中几乎缺乏关于如何在最常见分布假设下计算香农熵及KL散度的资源。本文通过利用贝叶斯网络的图结构，为这两种度量提出了计算高效的算法，并通过完整的数值示例进行说明。在此过程中，我们证明可将高斯贝叶斯网络的KL散度计算复杂度从三次方降至二次方。