The ability to compute the exact divergence between two high-dimensional distributions is useful in many applications but doing so naively is intractable. Computing the alpha-beta divergence -- a family of divergences that includes the Kullback-Leibler divergence and Hellinger distance -- between the joint distribution of two decomposable models, i.e chordal Markov networks, can be done in time exponential in the treewidth of these models. However, reducing the dissimilarity between two high-dimensional objects to a single scalar value can be uninformative. Furthermore, in applications such as supervised learning, the divergence over a conditional distribution might be of more interest. Therefore, we propose an approach to compute the exact alpha-beta divergence between any marginal or conditional distribution of two decomposable models. Doing so tractably is non-trivial as we need to decompose the divergence between these distributions and therefore, require a decomposition over the marginal and conditional distributions of these models. Consequently, we provide such a decomposition and also extend existing work to compute the marginal and conditional alpha-beta divergence between these decompositions. We then show how our method can be used to analyze distributional changes by first applying it to a benchmark image dataset. Finally, based on our framework, we propose a novel way to quantify the error in contemporary superconducting quantum computers. Code for all experiments is available at: https://lklee.dev/pub/2023-icdm/code
翻译:精确计算两个高维分布之间的散度在许多应用中非常有用,但直接计算往往是不可行的。计算两个可分解模型(即弦马尔可夫网络)联合分布之间的α-β散度(包括Kullback-Leibler散度和Hellinger距离的散度族)可以在这些模型树宽指数时间内完成。然而,将两个高维对象之间的差异简化为单一标量值可能缺乏信息量。此外,在监督学习等应用中,条件分布上的散度可能更具意义。因此,我们提出了一种方法,能够精确计算两个可分解模型任意边际或条件分布之间的α-β散度。实现这一点的可处理性并非易事,因为我们需要分解这些分布之间的散度,从而要求对这些模型的边际分布和条件分布进行分解。为此,我们提供了这样的分解,并扩展了现有工作,以计算这些分解之间的边际和条件α-β散度。然后,我们展示了如何通过首先将方法应用于基准图像数据集来分析分布变化。最后,基于我们的框架,我们提出了一种量化当代超导量子计算机中误差的新方法。所有实验的代码均可从 https://lklee.dev/pub/2023-icdm/code 获取。