Two-time-scale stochastic approximation, a generalized version of the popular stochastic approximation, has found broad applications in many areas including stochastic control, optimization, and machine learning. Despite of its popularity, theoretical guarantees of this method, especially its finite-time performance, are mostly achieved for the linear case while the results for the nonlinear counterpart are very sparse. Motivated by the classic control theory for singularly perturbed systems, we study in this paper the asymptotic convergence and finite-time analysis of the nonlinear two-time-scale stochastic approximation. Under some fairly standard assumptions, we provide a formula that characterizes the rate of convergence of the main iterates to the desired solutions. In particular, we show that the method achieves a convergence in expectation at a rate $\mathcal{O}(1/k^{2/3})$, where $k$ is the number of iterations. The key idea in our analysis is to properly choose the two step sizes to characterize the coupling between the fast and slow-time-scale iterates.

翻译：流行的随机近似(通用版的流行随机近似)在很多领域都得到了广泛的应用,包括随机控制、优化和机器学习。尽管它很受欢迎,但这一方法的理论保障,特别是其有限时间性能,大部分是线性案例的理论保障,而非线性对应方的结果则非常稀少。根据对奇特扰动系统的经典控制理论,我们在本文件中研究非线性双级随机近近近近的无时间趋同和有限时间分析。根据一些相当标准的假设,我们提供了一种公式,说明主要试样与理想解决办法的趋同率。特别是,我们表明,该方法达到了预期的趋同率($\mathcal{O}(1/k ⁇ 2/3})$,其中美元是迭代数。我们分析的关键思想是正确选择两步尺大小来描述快速和慢时相交的交点。