Explicit, momentum-based dynamics that optimize functions defined on Lie groups can be constructed via variational optimization and momentum trivialization. Structure preserving time discretizations can then turn this dynamics into optimization algorithms. This article investigates two types of discretization, Lie Heavy-Ball, which is a known splitting scheme, and Lie NAG-SC, which is newly proposed. Their convergence rates are explicitly quantified under $L$-smoothness and local strong convexity assumptions. Lie NAG-SC provides acceleration over the momentumless case, i.e. Riemannian gradient descent, but Lie Heavy-Ball does not. When compared to existing accelerated optimizers for general manifolds, both Lie Heavy-Ball and Lie NAG-SC are computationally cheaper and easier to implement, thanks to their utilization of group structure. Only gradient oracle and exponential map are required, but not logarithm map or parallel transport which are computational costly.
翻译:基于动量的显式动力学可通过变分优化与动量平凡化构建,用于优化定义在李群上的函数。结构保持的时间离散化可将此动力学转化为优化算法。本文研究了两种离散化方案:已知的分裂格式——李群重球法,以及新提出的李群NAG-SC。在$L$-光滑性与局部强凸性假设下,我们明确量化了它们的收敛速率。李群NAG-SC相较于无动量情形(即黎曼梯度下降法)实现了加速,而李群重球法则未表现出加速效果。与现有适用于一般流形的加速优化器相比,李群重球法与李群NAG-SC均因充分利用群结构而具有更低计算成本与更易实现的优势。二者仅需梯度预言机与指数映射,无需计算代价高昂的对数映射或平行移动。