Bregman divergences play a pivotal role in statistics, machine learning and computational information geometry. Particularly in the context of machine learning, they are central to clustering, exponential families, parameter estimation and optimisation, among other things. Despite this, the full toolkit of Hilbert spaces and in particular reproducing kernel Hilbert spaces have not been systematically developed and applied to functional Bregman divergences, where points are functions rather than finite-dimensional parameter vectors. While other types of functional Bregman divergences have been studied, these are typically in a Banach space rather than more directly aligned with kernel methods and Hilbert-space geometry commonly used in machine learning. We consider functional Bregman divergences on a Hilbert space, where the self-dual pairing and Riesz representer afford us particularly convenient calculus. Further specialising Bregman generators as a composition involving a kernel mean embedding makes such divergences easy to estimate. We discuss applications in clustering, universal estimation, robust estimation and generative modelling, and contrast our approach with other types of Bregman divergences.

翻译：布雷格曼散度在统计学、机器学习和计算信息几何中扮演着核心角色。尤其在机器学习领域，它们对聚类、指数族、参数估计和优化等问题至关重要。然而，希尔伯特空间的完整工具集——特别是再生核希尔伯特空间——尚未被系统地开发并应用于函数型布雷格曼散度（其中点被视为函数而非有限维参数向量）。尽管已有其他类型的函数型布雷格曼散度被研究，但它们通常基于巴拿赫空间，而非更直接地与机器学习中常用的核方法和希尔伯特空间几何对齐。本文研究希尔伯特空间上的函数型布雷格曼散度，其中自对偶配对和里斯表示定理为我们提供了特别便利的微积分工具。进一步将布雷格曼生成器特化为包含核均值嵌入的复合形式，使得此类散度易于估计。我们讨论了其在聚类、通用估计、鲁棒估计和生成建模中的应用，并将我们的方法与其他类型的布雷格曼散度进行了对比。