Hamilton and Moitra (2021) showed that, in certain regimes, it is not possible to accelerate Riemannian gradient descent in the hyperbolic plane if we restrict ourselves to algorithms which make queries in a (large) bounded domain and which receive gradients and function values corrupted by a (small) amount of noise. We show that acceleration remains unachievable for any deterministic algorithm which receives exact gradient and function-value information (unbounded queries, no noise). Our results hold for the classes of strongly and nonstrongly geodesically convex functions, and for a large class of Hadamard manifolds including hyperbolic spaces and the symmetric space $\mathrm{SL}(n) / \mathrm{SO}(n)$ of positive definite $n \times n$ matrices of determinant one. This cements a surprising gap between the complexity of convex optimization and geodesically convex optimization: for hyperbolic spaces, Riemannian gradient descent is optimal on the class of smooth and and strongly geodesically convex functions, in the regime where the condition number scales with the radius of the optimization domain. The key idea for proving the lower bound consists of perturbing the hard functions of Hamilton and Moitra (2021) with sums of bump functions chosen by a resisting oracle.
翻译:Hamilton和Moitra(2021)表明,在某些情形下,若限制算法仅在(大)有界域内查询,且接收被(小)噪声污染的梯度与函数值,则无法在双曲平面上加速黎曼梯度下降。我们证明:对于任何接收精确梯度与函数值信息(无界查询、无噪声)的确定性算法,加速仍然无法实现。我们的结论适用于强测地凸与非强测地凸函数类,以及一大类Hadamard流形,包括双曲空间和行列式为1的$n \times n$正定矩阵对称空间$\mathrm{SL}(n) / \mathrm{SO}(n)$。这巩固了凸优化与测地凸优化复杂度之间的惊人差距:在双曲空间上,当条件数随优化域半径缩放时,黎曼梯度下降对光滑且强测地凸函数类是最优的。证明下界的关键思想在于:通过由抵抗性预言机选择的凸函数和扰动Hamilton-Moitra(2021)的困难函数。