This paper delves into stochastic optimization problems that involve Markovian noise. We present a unified approach for the theoretical analysis of first-order gradient methods for stochastic optimization and variational inequalities. Our approach covers scenarios for both non-convex and strongly convex minimization problems. To achieve an optimal (linear) dependence on the mixing time of the underlying noise sequence, we use the randomized batching scheme, which is based on the multilevel Monte Carlo method. Moreover, our technique allows us to eliminate the limiting assumptions of previous research on Markov noise, such as the need for a bounded domain and uniformly bounded stochastic gradients. Our extension to variational inequalities under Markovian noise is original. Additionally, we provide lower bounds that match the oracle complexity of our method in the case of strongly convex optimization problems.

翻译：本文深入研究了涉及马尔可夫噪声的随机优化问题。我们提出了一种用于理论分析随机优化与变分不等式中一阶梯度方法的统一框架。该方法涵盖了非凸和强凸最小化问题的场景。为了实现对底层噪声序列混合时间的最优（线性）依赖关系，我们采用了基于多层蒙特卡洛方法的随机批处理方案。此外，我们的技术消除了以往关于马尔可夫噪声研究中的限制性假设，例如有界域和一致有界随机梯度的要求。我们关于马尔可夫噪声下变分不等式的推广具有原创性。同时，我们提供了在强凸优化问题场景下与所提方法预言机复杂度相匹配的下界。