Most commonly used $f$-divergences of measures, e.g., the Kullback-Leibler divergence, are subject to limitations regarding the support of the involved measures. A remedy consists of regularizing the $f$-divergence by a squared maximum mean discrepancy (MMD) associated with a characteristic kernel $K$. In this paper, we use the so-called kernel mean embedding to show that the corresponding regularization can be rewritten as the Moreau envelope of some function in the reproducing kernel Hilbert space associated with $K$. Then, we exploit well-known results on Moreau envelopes in Hilbert spaces to prove properties of the MMD-regularized $f$-divergences and, in particular, their gradients. Subsequently, we use our findings to analyze Wasserstein gradient flows of MMD-regularized $f$-divergences. Finally, we consider Wasserstein gradient flows starting from empirical measures and provide proof-of-the-concept numerical examples with Tsallis-$\alpha$ divergences.
翻译:最常用的度量f-散度(例如Kullback-Leibler散度)在涉及度量的支撑集时存在局限性。一种补救方案是通过与特征核K相关的平方最大均值差异(MMD)对f-散度进行正则化。本文利用核均值嵌入证明,相应的正则化可改写为再生核希尔伯特空间中某个函数(与核K相关)的Moreau包络。接着,我们利用希尔伯特空间中Moreau包络的已知结论,证明MMD正则化f-散度的性质,特别是其梯度。随后,我们应用这些发现分析MMD正则化f-散度的Wasserstein梯度流。最后,我们考虑从经验度量出发的Wasserstein梯度流,并利用Tsallis-α散度提供概念验证性数值示例。