In this paper, we study second-order algorithms for the convex-concave minimax problem, which has attracted much attention in many fields such as machine learning in recent years. We propose a Lipschitz-free cubic regularization (LF-CR) algorithm for solving the convex-concave minimax optimization problem without knowing the Lipschitz constant. It can be shown that the iteration complexity of the LF-CR algorithm to obtain an $\epsilon$-optimal solution with respect to the restricted primal-dual gap is upper bounded by $\mathcal{O}(\frac{\rho\|z^0-z^*\|^3}{\epsilon})^{\frac{2}{3}}$, where $z^0=(x^0,y^0)$ is a pair of initial points, $z^*=(x^*,y^*)$ is a pair of optimal solutions, and $\rho$ is the Lipschitz constant. We further propose a fully parameter-free cubic regularization (FF-CR) algorithm that does not require any parameters of the problem, including the Lipschitz constant and the upper bound of the distance from the initial point to the optimal solution. We also prove that the iteration complexity of the FF-CR algorithm to obtain an $\epsilon$-optimal solution with respect to the gradient norm is upper bounded by $\mathcal{O}(\frac{\rho\|z^0-z^*\|^2}{\epsilon})^{\frac{2}{3}}$. Numerical experiments show the efficiency of both algorithms. To the best of our knowledge, the proposed FF-CR algorithm is the first completely parameter-free second-order algorithm for solving convex-concave minimax optimization problems, and its iteration complexity is consistent with the optimal iteration complexity lower bound of existing second-order algorithms with parameters for solving convex-concave minimax problems.

翻译：本文研究凸凹极小极大问题的二阶算法，该问题近年来在机器学习等诸多领域备受关注。我们提出一种无需已知Lipschitz常数的Lipschitz无关三次正则化（LF-CR）算法来求解凸凹极小极大优化问题。可以证明，LF-CR算法获得关于限制原始-对偶间隙的$\epsilon$-最优解的迭代复杂度上界为$\mathcal{O}(\frac{\rho\|z^0-z^*\|^3}{\epsilon})^{\frac{2}{3}}$，其中$z^0=(x^0,y^0)$为初始点对，$z^*=(x^*,y^*)$为最优解对，$\rho$为Lipschitz常数。我们进一步提出完全无需参数的三次正则化（FF-CR）算法，该算法不依赖问题的任何参数，包括Lipschitz常数以及初始点到最优解距离的上界。同时证明FF-CR算法获得关于梯度范数的$\epsilon$-最优解的迭代复杂度上界为$\mathcal{O}(\frac{\rho\|z^0-z^*\|^2}{\epsilon})^{\frac{2}{3}}$。数值实验验证了两种算法的有效性。据我们所知，所提出的FF-CR算法是首个完全无需参数且用于求解凸凹极小极大优化问题的二阶算法，其迭代复杂度与现有含参数二阶算法求解凸凹极小极大问题的最优迭代复杂度下界保持一致。