The projection operation is a critical component in a wide range of optimization algorithms, such as online gradient descent (OGD), for enforcing constraints and achieving optimal regret bounds. However, it suffers from computational complexity limitations in high-dimensional settings or when dealing with ill-conditioned constraint sets. Projection-free algorithms address this issue by replacing the projection oracle with more efficient optimization subroutines. But to date, these methods have been developed primarily in the Euclidean setting, and while there has been growing interest in optimization on Riemannian manifolds, there has been essentially no work in trying to utilize projection-free tools here. An apparent issue is that non-trivial affine functions are generally non-convex in such domains. In this paper, we present methods for obtaining sub-linear regret guarantees in online geodesically convex optimization on curved spaces for two scenarios: when we have access to (a) a separation oracle or (b) a linear optimization oracle. For geodesically convex losses, and when a separation oracle is available, our algorithms achieve $O(T^{1/2}\:)$ and $O(T^{3/4}\;)$ adaptive regret guarantees in the full information setting and the bandit setting, respectively. When a linear optimization oracle is available, we obtain regret rates of $O(T^{3/4}\;)$ for geodesically convex losses and $O(T^{2/3}\; log T )$ for strongly geodesically convex losses
翻译:投影操作是多种优化算法(如在线梯度下降法OGD)中强制执行约束并实现最优遗憾界的关键组成部分。然而,在高维设置或处理病态约束集时,其计算复杂度存在局限性。无投影算法通过将投影操作替换为更高效的优化子程序来解决此问题。但迄今为止,这些方法主要基于欧几里得空间开发,尽管对黎曼流形上优化的兴趣与日俱增,但在利用无投影工具方面仍基本处于空白状态。一个明显问题是:在该领域中,非平凡仿射函数通常不是凸函数。本文针对弯曲空间上的在线测地凸优化,提出了两种场景下实现次线性遗憾保证的方法:(a)可获取分离预言机时;(b)可获取线性优化预言机时。对于测地凸损失函数,在可获取分离预言机的情况下,我们的算法在完全信息设置和赌博机设置中分别实现了$O(T^{1/2})$和$O(T^{3/4})$的自适应遗憾保证。当可获取线性优化预言机时,我们对测地凸损失函数获得了$O(T^{3/4})$的遗憾率,对强测地凸损失函数获得了$O(T^{2/3}\log T)$的遗憾率。