This paper introduces a novel approach to probabilistic deep learning, kernel density matrices, which provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables. In quantum mechanics, a density matrix is the most general way to describe the state of a quantum system. This work extends the concept of density matrices by allowing them to be defined in a reproducing kernel Hilbert space. This abstraction allows the construction of differentiable models for density estimation, inference, and sampling, and enables their integration into end-to-end deep neural models. In doing so, we provide a versatile representation of marginal and joint probability distributions that allows us to develop a differentiable, compositional, and reversible inference procedure that covers a wide range of machine learning tasks, including density estimation, discriminative learning, and generative modeling. The broad applicability of the framework is illustrated by two examples: an image classification model that can be naturally transformed into a conditional generative model, and a model for learning with label proportions that demonstrates the framework's ability to deal with uncertainty in the training samples.

翻译：本文提出了一种概率深度学习的新方法——核密度矩阵，该方法提供了一种更简单而有效的机制来表示连续和离散随机变量的联合概率分布。在量子力学中，密度矩阵是描述量子系统状态的最一般方式。本研究将密度矩阵的概念扩展至再生核希尔伯特空间，使其能够在该空间中定义。这种抽象化使得构建用于密度估计、推理和采样的可微分模型成为可能，并能将其集成到端到端的深度神经模型中。由此，我们提供了一种灵活表示边缘概率分布和联合概率分布的方法，进而开发出可微分、可组合且可逆的推理流程，该流程覆盖了密度估计、判别学习与生成建模等广泛的机器学习任务。该框架的广泛适用性通过两个示例得以展示：一个是能够自然转化为条件生成模型的图像分类模型，另一个是用于标签比例学习的模型，该模型展示了框架处理训练样本不确定性的能力。