We propose an optimization algorithm for Variational Inference (VI) in complex models. Our approach relies on natural gradient updates where the variational space is a Riemann manifold. We develop an efficient algorithm for Gaussian Variational Inference that implicitly satisfies the positive definite constraint on the variational covariance matrix. Our Exact manifold Gaussian Variational Bayes (EMGVB) provides exact but simple update rules and is straightforward to implement. Due to its black-box nature, EMGVB stands as a ready-to-use solution for VI in complex models. Over five datasets, we empirically validate our feasible approach on different statistical, econometric, and deep learning models, discussing its performance with respect to baseline methods.

翻译：我们提出了一种针对复杂模型变分推断（VI）的优化算法。该方法基于自然梯度更新，其中变分空间为黎曼流形。本研究为高斯变分推断开发了一种高效算法，该算法隐式满足变分协方差矩阵的正定约束。所提出的精确流形高斯变分贝叶斯（EMGVB）提供了精确且简洁的更新规则，易于实现。凭借其黑箱特性，EMGVB可作为复杂模型中VI的即用型解决方案。通过在五个数据集上对统计、计量经济学及深度学习模型进行实证验证，我们讨论了该方法相较于基线方法的性能表现。