The monotone Variational Inequality (VI) is a general model with important applications in various engineering and scientific domains. In numerous instances, the VI problems are accompanied by function constraints that can be data-driven, making the usual projection operator challenging to compute. This paper presents novel first-order methods for the function-constrained Variational Inequality (FCVI) problem in smooth or nonsmooth settings with possibly stochastic operators and constraints. We introduce the AdOpEx method, which employs an operator extrapolation on the KKT operator of the FCVI in a smooth deterministic setting. Since this operator is not uniformly Lipschitz continuous in the Lagrange multipliers, we employ an adaptive two-timescale algorithm leading to bounded multipliers and achieving the optimal $O(1/T)$ convergence rate. For the nonsmooth and stochastic VIs, we introduce design changes to the AdOpEx method and propose a novel P-OpEx method that takes partial extrapolation. It converges at the rate of $O(1/\sqrt{T})$ when both the operator and constraints are stochastic or nonsmooth. This method has suboptimal dependence on the noise and Lipschitz constants of function constraints. We propose a constraint extrapolation approach leading to the OpConEx method that improves this dependence by an order of magnitude. All our algorithms easily extend to saddle point problems with function constraints that couple the primal and dual variables while maintaining the same complexity results. To the best of our knowledge, all our complexity results are new in the literature
翻译:单调变分不等式(VI)是一个通用模型,在众多工程与科学领域具有重要应用。在许多情况下,变分不等式问题伴随着可能由数据驱动的函数约束,这使得通常的投影算子难以计算。本文针对光滑或非光滑情形下可能具有随机算子与约束的函数约束变分不等式(FCVI)问题,提出了新颖的一阶方法。我们首先介绍了AdOpEx方法,该方法在光滑确定性设定下对FCVI的KKT算子采用算子外推技术。由于该算子在拉格朗日乘子上不具备一致Lipschitz连续性,我们采用自适应双时间尺度算法以获得有界乘子,并达到最优的$O(1/T)$收敛速率。针对非光滑随机变分不等式,我们对AdOpEx方法进行设计调整,提出了一种采用部分外推的新型P-OpEx方法。当算子与约束均为随机或非光滑时,该方法以$O(1/\sqrt{T})$速率收敛,但其在噪声与函数约束Lipschitz常数上的依赖关系未达最优。我们进一步提出约束外推方法,由此发展的OpConEx方法将该依赖关系改善了一个数量级。我们所有的算法均可自然扩展到具有耦合原始变量与对偶变量的函数约束鞍点问题,同时保持相同的复杂度结果。据我们所知,本文所有的复杂度结论在现有文献中均属首次提出。