We propose a new family of regularized R\'enyi divergences parametrized not only by the order $\alpha$ but also by a variational function space. These new objects are defined by taking the infimal convolution of the standard R\'enyi divergence with the integral probability metric (IPM) associated with the chosen function space. We derive a novel dual variational representation that can be used to construct numerically tractable divergence estimators. This representation avoids risk-sensitive terms and therefore exhibits lower variance, making it well-behaved when $\alpha>1$; this addresses a notable weakness of prior approaches. We prove several properties of these new divergences, showing that they interpolate between the classical R\'enyi divergences and IPMs. We also study the $\alpha\to\infty$ limit, which leads to a regularized worst-case-regret and a new variational representation in the classical case. Moreover, we show that the proposed regularized R\'enyi divergences inherit features from IPMs such as the ability to compare distributions that are not absolutely continuous, e.g., empirical measures and distributions with low-dimensional support. We present numerical results on both synthetic and real datasets, showing the utility of these new divergences in both estimation and GAN training applications; in particular, we demonstrate significantly reduced variance and improved training performance.

翻译：我们提出了一类新的正则化Rényi散度族，该类散度不仅由阶数$\alpha$参数化，还由变分函数空间参数化。这些新对象通过将标准Rényi散度与所选函数空间相关联的积分概率度量（IPM）进行下确界卷积而定义。我们推导了一种新颖的对偶变分表示，可用于构造数值上易于处理的散度估计器。该表示避免了风险敏感项，因此表现出更低的方差，并在$\alpha>1$时具有良好的性质；这解决了先前方法的一个显著缺陷。我们证明了这些新散度的若干性质，表明它们在经典Rényi散度和IPM之间进行插值。我们还研究了$\alpha\to\infty$的极限情况，该极限导致正则化的最坏情况遗憾，并在经典情形下提供了新的变分表示。此外，我们表明所提出的正则化Rényi散度继承了IPM的特征，例如能够比较并非绝对连续的分布，例如经验测度和具有低维支撑的分布。我们在合成数据集和真实数据集上呈现了数值结果，展示了这些新散度在估计和生成对抗网络（GAN）训练应用中的实用性；特别地，我们证明了显著降低的方差和提升的训练性能。