Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide-and-conquering them into a sparse collection of smaller ones; underlies Judea Pearl's causality; and determines their explainability and interpretability. Despite their popularity, there are few resources in the literature on how to compute Shannon's entropy and the Kullback-Leibler (KL) divergence for BNs under their most common distributional assumptions. In this paper, we provide computationally efficient algorithms for both by leveraging BNs' graphical structure, and we illustrate them with a complete set of numerical examples. In the process, we show it is possible to reduce the computational complexity of KL from cubic to quadratic for Gaussian BNs.

翻译：贝叶斯网络是机器学习和因果推断中的基础模型。其图形结构能够处理高维问题，将其分解为稀疏的小规模问题集合；这一结构支撑了朱迪亚·珀尔的因果理论，并决定了模型的可解释性与可理解性。尽管贝叶斯网络应用广泛，但现有文献中关于如何在其最常见分布假设下计算香农熵和KL散度的资源较少。本文通过利用贝叶斯网络的图形结构，提出了两种计算高效的算法，并通过完整的数值算例加以说明。在此过程中，我们证明了对于高斯贝叶斯网络，KL散度的计算复杂度可从立方阶降至平方阶。