We propose an optimization algorithm for Variational Inference (VI) in complex models. Our approach relies on natural gradient updates where the variational space is a Riemann manifold. We develop an efficient algorithm for Gaussian Variational Inference that implicitly satisfies the positive definite constraint on the variational covariance matrix. Our Exact manifold Gaussian Variational Bayes (EMGVB) provides exact but simple update rules and is straightforward to implement. Due to its black-box nature, EMGVB stands as a ready-to-use solution for VI in complex models. Over five datasets, we empirically validate our feasible approach on different statistical, econometric, and deep learning models, discussing its performance with respect to baseline methods.

翻译：我们提出一种用于复杂模型中变分推断（VI）的优化算法。该方法基于自然梯度更新，其中变分空间为黎曼流形。我们开发了一种高效的变分推断算法，该算法能隐式满足变分协方差矩阵的正定约束条件。我们的精确流形高斯变分贝叶斯方法（EMGVB）提供了精确且简单的更新规则，易于实现。由于具有黑箱特性，EMGVB可作为复杂模型中变分推断的即用型解决方案。我们在五个数据集上，针对不同统计、计量经济学及深度学习模型进行了实证验证，并讨论了其相较于基线方法的性能表现。