Single-call stochastic extragradient methods, like stochastic past extragradient (SPEG) and stochastic optimistic gradient (SOG), have gained a lot of interest in recent years and are one of the most efficient algorithms for solving large-scale min-max optimization and variational inequalities problems (VIP) appearing in various machine learning tasks. However, despite their undoubted popularity, current convergence analyses of SPEG and SOG require a bounded variance assumption. In addition, several important questions regarding the convergence properties of these methods are still open, including mini-batching, efficient step-size selection, and convergence guarantees under different sampling strategies. In this work, we address these questions and provide convergence guarantees for two large classes of structured non-monotone VIPs: (i) quasi-strongly monotone problems (a generalization of strongly monotone problems) and (ii) weak Minty variational inequalities (a generalization of monotone and Minty VIPs). We introduce the expected residual condition, explain its benefits, and show how it can be used to obtain a strictly weaker bound than previously used growth conditions, expected co-coercivity, or bounded variance assumptions. Equipped with this condition, we provide theoretical guarantees for the convergence of single-call extragradient methods for different step-size selections, including constant, decreasing, and step-size-switching rules. Furthermore, our convergence analysis holds under the arbitrary sampling paradigm, which includes importance sampling and various mini-batching strategies as special cases.

翻译：单步随机外梯度方法，如随机过去外梯度（SPEG）和随机乐观梯度（SOG），近年来受到广泛关注，是解决大规模极小极大优化和变分不等式问题（VIP）的最高效算法之一，这些问题普遍出现在各类机器学习任务中。然而，尽管这类方法广受欢迎，现有关于SPEG和SOG的收敛性分析均需假设方差有界。此外，关于这些方法收敛特性的若干重要问题仍悬而未解，包括小批量处理、高效步长选择以及不同采样策略下的收敛保证。本研究针对上述问题，为两类大规模结构化非单调VIP提供了收敛性保证：（i）拟强单调问题（强单调问题的推广）和（ii）弱敏蒂变分不等式（单调及敏蒂VIP的推广）。我们引入了期望残差条件，阐释其优势，并展示了该条件如何导出比此前使用的增长条件、期望余强制性或方差有界假设更严格的弱化界。基于该条件，我们为单步外梯度方法在多种步长选择（包括常数步长、递减步长和步长切换规则）下的收敛性提供了理论保证。此外，我们的收敛性分析适用于任意采样范式，其中包括重要性采样及各类小批量策略作为特例。