We propose and analyze several inexact regularized Newton-type methods for finding a global saddle point of convex-concave unconstrained min-max optimization problems. Compared to first-order methods, our understanding of second-order methods for min-max optimization is relatively limited, as obtaining global rates of convergence with second-order information can be much more involved. In this paper, we examine how second-order information is used to speed up extra-gradient methods, even under inexactness. In particular, we show that the proposed methods generate iterates that remain within a bounded set and that the averaged iterates converge to an $ε$-saddle point within $O(ε^{-2/3})$ iterations in terms of a restricted gap function. We also provide a simple routine for solving the subproblem at each iteration, requiring a single Schur decomposition and $O(\log\log(1/ε))$ calls to a linear system solver in a quasi-upper-triangular system. Thus, our method improves the existing line-search-based second-order min-max optimization methods by shaving off an $O(\log\log(1/ε))$ factor in the required number of Schur decompositions. Finally, we evaluate our method on both synthetic benchmarks and a real-world application arising from AUC maximization on standard LIBSVM datasets, and find that the proposed second-order approach delivers stronger practical efficiency than representative first-order methods on these problems.

翻译：我们提出并分析了几种非精确正则化牛顿型方法，用于求解凸-凹无约束极小极大优化问题的全局鞍点。与一阶方法相比，我们对二阶方法在极小极大优化中的理解相对有限，因为利用二阶信息获得全局收敛率可能更加复杂。本文研究了即使在非精确条件下，如何利用二阶信息加速外梯度方法。具体而言，我们证明所提出的方法生成的迭代点始终保持在有界集内，且平均迭代点关于限制间隙函数在$O(ε^{-2/3})$次迭代内收敛到$ε$-鞍点。我们还提出了一种简单的子问题求解例程，每次迭代仅需一次舒尔分解和$O(\log\log(1/ε))$次拟上三角线性系统求解器调用。因此，我们的方法将现有基于线搜索的二阶极小极大优化方法所需的舒尔分解次数降低了$O(\log\log(1/ε))$因子。最后，我们在合成基准测试和来自标准LIBSVM数据集上AUC最大化的实际应用中评估了该方法，发现所提出的二阶方法在这些问题上比代表性的一阶方法具有更强的实际效率。