Optimization tasks are crucial in statistical machine learning. Recently, there has been great interest in leveraging tools from dynamical systems to derive accelerated and robust optimization methods via suitable discretizations of continuous-time systems. However, these ideas have mostly been limited to Euclidean spaces and unconstrained settings, or to Riemannian gradient flows. In this work, we propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems over smooth manifolds, including problems with nonlinear constraints. We develop geometric/symplectic numerical integrators on manifolds that are "rate-matching," i.e., preserve the continuous-time rates of convergence. In particular, we introduce a dissipative RATTLE integrator able to achieve optimal convergence rate locally. Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.

翻译：优化任务在统计机器学习中至关重要。近年来，利用动力系统工具通过连续时间系统的适当离散化来推导加速且鲁棒的优化方法引起了广泛兴趣。然而，这些思想主要局限于欧几里得空间和无约束设置，或仅适用于黎曼梯度流。本文提出一种耗散扩展的狄拉克约束哈密顿系统理论，作为在光滑流形（包括含非线性约束的问题）上求解优化问题的通用框架。我们开发了流形上具有"速率匹配"特性的几何/辛数值积分器，即能够保持连续时间收敛速率。特别地，我们引入了一种耗散型RATTLE积分器，能够在局部实现最优收敛速率。我们提出的（加速）算法类不仅简单高效，而且适用于广泛的应用场景。