In this paper, we study a class of non-smooth non-convex problems in the form of $\min_{x}[\max_{y\in Y}\phi(x, y) - \max_{z\in Z}\psi(x, z)]$, where both $\Phi(x) = \max_{y\in Y}\phi(x, y)$ and $\Psi(x)=\max_{z\in Z}\psi(x, z)$ are weakly convex functions, and $\phi(x, y), \psi(x, z)$ are strongly concave functions in terms of $y$ and $z$, respectively. It covers two families of problems that have been studied but are missing single-loop stochastic algorithms, i.e., difference of weakly convex functions and weakly convex strongly-concave min-max problems. We propose a stochastic Moreau envelope approximate gradient method dubbed SMAG, the first single-loop algorithm for solving these problems, and provide a state-of-the-art non-asymptotic convergence rate. The key idea of the design is to compute an approximate gradient of the Moreau envelopes of $\Phi, \Psi$ using only one step of stochastic gradient update of the primal and dual variables. Empirically, we conduct experiments on positive-unlabeled (PU) learning and partial area under ROC curve (pAUC) optimization with an adversarial fairness regularizer to validate the effectiveness of our proposed algorithms.

翻译：本文研究一类非光滑非凸问题，其形式为 $\min_{x}[\max_{y\in Y}\phi(x, y) - \max_{z\in Z}\psi(x, z)]$，其中 $\Phi(x) = \max_{y\in Y}\phi(x, y)$ 与 $\Psi(x)=\max_{z\in Z}\psi(x, z)$ 均为弱凸函数，且 $\phi(x, y)$ 和 $\psi(x, z)$ 分别关于 $y$ 和 $z$ 是强凹函数。该形式涵盖了两类已被研究但尚缺单循环随机算法的问题族：弱凸函数之差问题与弱凸强凹极小-极大问题。我们提出了一种名为SMAG的随机莫罗包络近似梯度法，这是首个用于求解此类问题的单循环算法，并提供了当前最优的非渐近收敛速率。设计的关键思想是仅通过对原始变量和对偶变量进行一次随机梯度更新，来计算 $\Phi$ 和 $\Psi$ 的莫罗包络的近似梯度。在实证研究中，我们在正例-无标记（PU）学习和带有对抗公平正则化的部分ROC曲线下面积（pAUC）优化任务上进行了实验，以验证所提算法的有效性。