This work introduces a novel probabilistic deep learning technique called deep Gaussian mixture ensembles (DGMEs), which enables accurate quantification of both epistemic and aleatoric uncertainty. By assuming the data generating process follows that of a Gaussian mixture, DGMEs are capable of approximating complex probability distributions, such as heavy-tailed or multimodal distributions. Our contributions include the derivation of an expectation-maximization (EM) algorithm used for learning the model parameters, which results in an upper-bound on the log-likelihood of training data over that of standard deep ensembles. Additionally, the proposed EM training procedure allows for learning of mixture weights, which is not commonly done in ensembles. Our experimental results demonstrate that DGMEs outperform state-of-the-art uncertainty quantifying deep learning models in handling complex predictive densities.

翻译：本文提出了一种名为深度高斯混合集成（DGMEs）的新型概率深度学习技术，能够同时精确量化认知不确定性与偶然不确定性。通过假设数据生成过程遵循高斯混合分布，DGMEs可近似复杂概率分布（如重尾分布或多峰分布）。我们的贡献包括推导出一种用于学习模型参数的期望最大化（EM）算法，该算法在训练数据的对数似然上界方面优于标准深度集成。此外，所提出的EM训练过程允许学习混合权重，这在集成方法中并不常见。实验结果表明，DGMEs在处理复杂预测密度方面优于当前最先进的深度不确定性量化模型。