Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide them into a sparse collection of smaller ones, underlies Judea Pearl's causality, and determines their explainability and interpretability. Despite their popularity, there are almost no resources in the literature on how to compute Shannon's entropy and the Kullback-Leibler (KL) divergence for BNs under their most common distributional assumptions. In this paper, we provide computationally efficient algorithms for both by leveraging BNs' graphical structure, and we illustrate them with a complete set of numerical examples. In the process, we show it is possible to reduce the computational complexity of KL from cubic to quadratic for Gaussian BNs.

翻译：贝叶斯网络（BNs）是机器学习和因果推断中的基础模型。其图结构可处理高维问题，将其分解为稀疏的小规模子问题，这支撑了Judea Pearl的因果关系理论，并决定了模型的可解释性。尽管应用广泛，现有文献中几乎缺乏关于在常见分布假设下计算贝叶斯网络香农熵与KL散度的资源。本文通过利用贝叶斯网络的图结构，提出了两种高效算法，并辅以完整数值算例进行说明。在此过程中，我们证明对高斯贝叶斯网络，可将KL散度的计算复杂度从三次方降至二次方。