Stochastic approximation (SA) is a method for finding the root of an operator perturbed by noise. There is a rich literature establishing the asymptotic normality of rescaled SA iterates under fairly mild conditions. However, these asymptotic results do not quantify the accuracy of the Gaussian approximation in finite time. In this paper, we establish explicit non-asymptotic bounds on the Wasserstein distance between the distribution of the rescaled iterate at time k and the asymptotic Gaussian limit for various choices of step-sizes including constant and polynomially decaying. As an immediate consequence, we obtain tail bounds on the error of SA iterates at any time. We obtain the sharp rates by first studying the convergence rate of the discrete Ornstein-Uhlenbeck (O-U) process driven by general noise, whose stationary distribution is identical to the limiting Gaussian distribution of the rescaled SA iterates. We believe that this is of independent interest, given its connection to sampling literature. The analysis involves adapting Stein's method for Gaussian approximation to handle the matrix weighted sum of i.i.d. random variables. The desired finite-time bounds for SA are obtained by characterizing the error dynamics between the rescaled SA iterate and the discrete time O-U process and combining it with the convergence rate of the latter process.

翻译：随机逼近（SA）是一种在噪声扰动下寻找算子根的方法。现有大量文献在相当温和的条件下建立了重标度SA迭代的渐近正态性。然而，这些渐近结果并未量化有限时间内高斯近似的精度。本文针对包括常数步长和多项式衰减步长在内的多种步长选择，建立了第k时刻重标度迭代分布与渐近高斯极限之间Wasserstein距离的显式非渐近界。作为直接推论，我们得到了任意时刻SA迭代误差的尾部边界。我们首先研究由一般噪声驱动的离散Ornstein-Uhlenbeck（O-U）过程的收敛速率（其平稳分布与重标度SA迭代的极限高斯分布相同），从而获得了最优收敛率。考虑到该方法与采样文献的关联性，我们认为这一研究具有独立价值。分析过程涉及将Stein高斯近似方法适配于处理独立同分布随机变量的矩阵加权和。通过刻画重标度SA迭代与离散时间O-U过程之间的误差动态，并将其与后者的收敛速率相结合，最终获得了SA所需的有限时间界。