We derive a new variational formula for the R{\'e}nyi family of divergences, $R_\alpha(Q\|P)$, between probability measures $Q$ and $P$. Our result generalizes the classical Donsker-Varadhan variational formula for the Kullback-Leibler divergence. We further show that this R{\'e}nyi variational formula holds over a range of function spaces; this leads to a formula for the optimizer under very weak assumptions and is also key in our development of a consistency theory for R{\'e}nyi divergence estimators. By applying this theory to neural-network estimators, we show that if a neural network family satisfies one of several strengthened versions of the universal approximation property then the corresponding R{\'e}nyi divergence estimator is consistent. In contrast to density-estimator based methods, our estimators involve only expectations under $Q$ and $P$ and hence are more effective in high dimensional systems. We illustrate this via several numerical examples of neural network estimation in systems of up to 5000 dimensions.

翻译：我们得出了一个新变式公式, 用于R'e'neny 差异的大家庭, $R'alpha( ⁇ P), 概率计量 $Q 和 $P 。 我们的结果将古典的Donsker- Varadhan 变式公式概括为 Kullback- Leiber 差异。 我们进一步显示, 这个R' e}nyi 变式公式维持着一系列功能空间; 这导致在非常薄弱的假设下优化一个公式, 也是我们为 R' e}ny 差异估测员制定一致性理论的关键。 通过将这一理论应用于神经网络估测器, 我们显示, 如果神经网络组满足了几个强化版本的通用近似特性之一, 那么相应的R' e}nyi 差异估测器是一致的。 与基于密度估测器的方法相比, 我们的估测器只包含低于$和$P$P$的预期值, 因而在高维度系统中更为有效。 我们通过数个数字例子来说明这一点, 我们通过到 5000 维的系统中的神经网络估算。