We consider the problem of minimizing a non-convex function over a smooth manifold $\mathcal{M}$. We propose a novel algorithm, the Orthogonal Directions Constrained Gradient Method (ODCGM) which only requires computing a projection onto a vector space. ODCGM is infeasible but the iterates are constantly pulled towards the manifold, ensuring the convergence of ODCGM towards $\mathcal{M}$. ODCGM is much simpler to implement than the classical methods which require the computation of a retraction. Moreover, we show that ODCGM exhibits the near-optimal oracle complexities $\mathcal{O}(1/\varepsilon^2)$ and $\mathcal{O}(1/\varepsilon^4)$ in the deterministic and stochastic cases, respectively. Furthermore, we establish that, under an appropriate choice of the projection metric, our method recovers the landing algorithm of Ablin and Peyr\'e (2022), a recently introduced algorithm for optimization over the Stiefel manifold. As a result, we significantly extend the analysis of Ablin and Peyr\'e (2022), establishing near-optimal rates both in deterministic and stochastic frameworks. Finally, we perform numerical experiments which shows the efficiency of ODCGM in a high-dimensional setting.

翻译：我们考虑在光滑流形 $\mathcal{M}$ 上最小化非凸函数的问题。本文提出了一种新颖算法——正交方向约束梯度法（ODCGM），该方法仅需计算到向量空间的投影。ODCGM 虽不可行，但其迭代点持续被拉向流形，从而确保 ODCGM 收敛到 $\mathcal{M}$。与需要计算收缩的传统方法相比，ODCGM 的实现更为简单。此外，我们证明了在确定性和随机情形下，ODCGM 分别具有接近最优的 oracle 复杂度 $\mathcal{O}(1/\varepsilon^2)$ 和 $\mathcal{O}(1/\varepsilon^4)$。进一步，我们表明，通过适当选择投影度量，我们的方法可恢复 Ablin 和 Peyré（2022）提出的着陆算法（一种最近提出的用于施蒂费尔流形优化的算法）。因此，本文显著扩展了 Ablin 和 Peyré（2022）的分析，在确定性和随机框架下均建立了接近最优的收敛速率。最后，我们进行了数值实验，展示了 ODCGM 在高维场景中的效率。