Although Deep Learning (DL) has achieved success in complex Artificial Intelligence (AI) tasks, it suffers from various notorious problems (e.g., feature redundancy, and vanishing or exploding gradients), since updating parameters in Euclidean space cannot fully exploit the geometric structure of the solution space. As a promising alternative solution, Riemannian-based DL uses geometric optimization to update parameters on Riemannian manifolds and can leverage the underlying geometric information. Accordingly, this article presents a comprehensive survey of applying geometric optimization in DL. At first, this article introduces the basic procedure of the geometric optimization, including various geometric optimizers and some concepts of Riemannian manifold. Subsequently, this article investigates the application of geometric optimization in different DL networks in various AI tasks, e.g., convolution neural network, recurrent neural network, transfer learning, and optimal transport. Additionally, typical public toolboxes that implement optimization on manifold are also discussed. Finally, this article makes a performance comparison between different deep geometric optimization methods under image recognition scenarios.

翻译：尽管深度学习（DL）在复杂人工智能（AI）任务中取得了成功，但因其在欧几里得空间更新参数无法充分利用解空间的几何结构，仍面临各种棘手问题（例如特征冗余、梯度消失或爆炸）。作为一种具有前景的替代方案，基于黎曼的深度学习通过几何优化在黎曼流形上更新参数，能够利用潜在的几何信息。据此，本文对几何优化在深度学习中的应用进行了全面综述。首先，本文介绍了几何优化的基本流程，包括各类几何优化器及黎曼流形的相关概念。随后，本文研究了几何优化在不同AI任务中各类深度学习网络中的应用，例如卷积神经网络、循环神经网络、迁移学习和最优传输。此外，还讨论了几种常用的流形优化公共工具箱。最后，本文在图像识别场景下对不同深度几何优化方法进行了性能比较。