The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precisely the mutual information between these random variables. In this paper, we analytically describe the profiles of multivariate normal distributions and introduce a novel family of distributions, Bend and Mix Models, for which the profile can be accurately estimated using Monte Carlo methods. We then show how Bend and Mix Models can be used to study the limitations of existing mutual information estimators, investigate the behavior of neural critics used in variational estimators, and understand the effect of experimental outliers on mutual information estimation. Finally, we show how Bend and Mix Models can be used to obtain model-based Bayesian estimates of mutual information, suitable for problems with available domain expertise in which uncertainty quantification is necessary.

翻译：点互信息剖面（简称剖面）是指给定一对随机变量的点互信息分布。其重要性质之一是期望值恰好等于这些随机变量间的互信息。本文解析描述了多元正态分布的剖面，并引入了一类新颖的分布族——弯曲混合模型，该类模型的剖面可通过蒙特卡洛方法进行精确估计。随后，我们展示了如何利用弯曲混合模型来研究现有互信息估计器的局限性、探究变分估计器中神经判别器的行为特性，以及分析实验异常值对互信息估计的影响。最后，我们论证了弯曲混合模型如何用于获得基于模型的贝叶斯互信息估计，该方法适用于具备领域先验知识且需进行不确定性量化的研究问题。