The Stochastic Approximation (SA) algorithm introduced by Robbins and Monro in 1951 has been a standard method for solving equations of the form $\mathbf{f}({\boldsymbol {\theta}}) = \mathbf{0}$, when only noisy measurements of $\mathbf{f}(\cdot)$ are available. If $\mathbf{f}({\boldsymbol {\theta}}) = \nabla J({\boldsymbol {\theta}})$ for some function $J(\cdot)$, then SA can also be used to find a stationary point of $J(\cdot)$. At each time $t$, the current guess ${\boldsymbol {\theta}}_t$ is updated to ${\boldsymbol {\theta}}_{t+1}$ using a noisy measurement of the form $\mathbf{f}({\boldsymbol {\theta}}_t) + {\boldsymbol {\xi}}_{t+1}$. In much of the literature, it is assumed that the error term ${\boldsymbol {\xi}}_{t+1}$ has zero conditional mean, and/or that its conditional variance is bounded as a function of $t$ (though not necessarily with respect to ${\boldsymbol {\theta}}_t$). Over the years, SA has been applied to a variety of areas, out of which the focus in this paper is on convex and nonconvex optimization. As it turns out, in these applications, the above-mentioned assumptions on the measurement error do not always hold. In zero-order methods, the error neither has zero mean nor bounded conditional variance. In the present paper, we extend SA theory to encompass errors with nonzero conditional mean and/or unbounded conditional variance. In addition, we derive estimates for the rate of convergence of the algorithm, and compute the ``optimal step size sequences'' to maximize the estimated rate of convergence.

翻译：由Robbins和Monro于1951年提出的随机逼近算法，是在仅能获得函数$\mathbf{f}(\cdot)$含噪测量值时求解方程$\mathbf{f}({\boldsymbol {\theta}}) = \mathbf{0}$的标准方法。若$\mathbf{f}({\boldsymbol {\theta}}) = \nabla J({\boldsymbol {\theta}})$对某函数$J(\cdot)$成立，则SA亦可用于寻找$J(\cdot)$的驻点。在每次迭代$t$中，当前估计值${\boldsymbol {\theta}}_t$通过含噪测量值$\mathbf{f}({\boldsymbol {\theta}}_t) + {\boldsymbol {\xi}}_{t+1}$更新为${\boldsymbol {\theta}}_{t+1}$。现有文献通常假设误差项${\boldsymbol {\xi}}_{t+1}$的条件均值为零，且/或其条件方差作为$t$的函数有界（但未必关于${\boldsymbol {\theta}}_t$一致有界）。多年来，SA已被应用于多个领域，而本文聚焦于其在凸优化与非凸优化中的应用。研究发现，在这些应用中，前述关于测量误差的假设并非始终成立。在零阶方法中，误差既不满足零均值条件，也不具有有界条件方差。本文通过扩展SA理论，使其能够处理具有非零条件均值和/或无界条件方差的误差。此外，我们推导了算法的收敛速率估计，并计算了最大化估计收敛速率的"最优步长序列"。