This article studies the Fisher-Rao gradient, also referred to as the natural gradient, of the evidence lower bound (ELBO) which plays a central role in generative machine learning. It reveals that the gap between the evidence and its lower bound, the ELBO, has essentially a vanishing natural gradient within unconstrained optimization. As a result, maximization of the ELBO is equivalent to minimization of the Kullback-Leibler divergence from a target distribution, the primary objective function of learning. Building on this insight, we derive a condition under which this equivalence persists even when optimization is constrained to a model. This condition yields a geometric characterization, which we formalize through the notion of a cylindrical model.

翻译：本文研究了证据下界（ELBO）的Fisher-Rao梯度（亦称自然梯度），该下界在生成式机器学习中具有核心作用。研究揭示：在无约束优化中，证据与其下界ELBO之间的差距本质上具有趋近于零的自然梯度。因此，最大化ELBO等价于最小化与目标分布之间的Kullback-Leibler散度——这正是学习过程的主要目标函数。基于这一发现，我们推导出即使在模型约束条件下该等价关系依然成立的条件。该条件可通过柱面模型的概念形成几何特征的形式化表述。