This paper introduces a method for efficiently approximating the inverse of the Fisher information matrix, a crucial step in achieving effective variational Bayes inference. A notable aspect of our approach is the avoidance of analytically computing the Fisher information matrix and its explicit inversion. Instead, we introduce an iterative procedure for generating a sequence of matrices that converge to the inverse of Fisher information. The natural gradient variational Bayes algorithm without analytic expression of the Fisher matrix and its inversion is provably convergent and achieves a convergence rate of order O(log s/s), with s the number of iterations. We also obtain a central limit theorem for the iterates. Implementation of our method does not require storage of large matrices, and achieves a linear complexity in the number of variational parameters. Our algorithm exhibits versatility, making it applicable across a diverse array of variational Bayes domains, including Gaussian approximation and normalizing flow Variational Bayes. We offer a range of numerical examples to demonstrate the efficiency and reliability of the proposed variational Bayes method.

翻译：本文介绍了一种高效近似Fisher信息矩阵逆的方法，这是实现有效变分贝叶斯推断的关键步骤。本方法的一个显著特点是避免了对Fisher信息矩阵的解析计算及其显式求逆。我们提出了一种迭代过程，可生成收敛到Fisher信息逆的矩阵序列。该无需Fisher矩阵解析表达式及其求逆的自然梯度变分贝叶斯算法被证明具有收敛性，且收敛速度达到O(log s/s)阶（其中s为迭代次数）。我们还得到了迭代量的中心极限定理。本方法的实现无需存储大型矩阵，且计算复杂度与变分参数数量呈线性关系。该算法具有广泛适用性，可应用于多种变分贝叶斯领域，包括高斯近似和归一化流变分贝叶斯。我们提供了一系列数值算例，以证明所提变分贝叶斯方法的有效性和可靠性。