In this paper, based a novel primal-dual dynamical model with adaptive scaling parameters and Bregman divergences, we propose new accelerated primal-dual proximal gradient splitting methods for solving bilinear saddle-point problems with provable optimal nonergodic convergence rates. For the first, using the spectral analysis, we show that a naive extension of acceleration model for unconstrained optimization problems to a quadratic game is unstable. Motivated by this, we present an accelerated primal-dual hybrid gradient (APDHG) flow which combines acceleration with careful velocity correction. To work with non-Euclidean distances, we also equip our APDHG model with general Bregman divergences and prove the exponential decay of a Lyapunov function. Then, new primal-dual splitting methods are developed based on proper semi-implicit Euler schemes of the continuous model, and the theoretical convergence rates are nonergodic and optimal with respect to the matrix norms,\, Lipschitz constants and convexity parameters. Thanks to the primal and dual scaling parameters, both the algorithm designing and convergence analysis cover automatically the convex and (partially) strongly convex objectives. Moreover, the use of Bregman divergences not only unifies the standard Euclidean distances and general cases in an elegant way, but also makes our methods more flexible and adaptive to problem-dependent metrics.
翻译:本文基于一种具有自适应缩放参数和Bregman散度的新型原始-对偶动力学模型,提出了求解双线性鞍点问题的新加速原始-对偶近端梯度分裂方法,并证明了其最优的非遍历收敛速率。首先,通过谱分析,我们证明了将无约束优化问题的加速模型直接推广到二次博弈是不稳定的。受此启发,我们提出了一种结合加速机制与精细速度修正的加速原始-对偶混合梯度(APDHG)流。为处理非欧几里得距离,我们还为APDHG模型配备了广义Bregman散度,并证明了Lyapunov函数的指数衰减性。随后,基于连续模型的适当半隐式欧拉格式,我们开发了新的原始-对偶分裂方法,其理论收敛速率相对于矩阵范数、Lipschitz常数和凸性参数具有非遍历性和最优性。得益于原始与对偶缩放参数,算法设计与收敛分析自动涵盖了凸目标及(部分)强凸目标情形。此外,Bregman散度的使用不仅以优雅方式统一了标准欧几里得距离与一般情形,还使我们的方法能更灵活地适应问题相关的度量结构。