Recent methods in geometric deep learning have introduced various neural networks to operate over data that lie on Riemannian manifolds. Such networks are often necessary to learn well over graphs with a hierarchical structure or to learn over manifold-valued data encountered in the natural sciences. These networks are often inspired by and directly generalize standard Euclidean neural networks. However, extending Euclidean networks is difficult and has only been done for a select few manifolds. In this work, we examine the residual neural network (ResNet) and show how to extend this construction to general Riemannian manifolds in a geometrically principled manner. Originally introduced to help solve the vanishing gradient problem, ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks. We find that our Riemannian ResNets mirror these desirable properties: when compared to existing manifold neural networks designed to learn over hyperbolic space and the manifold of symmetric positive definite matrices, we outperform both kinds of networks in terms of relevant testing metrics and training dynamics.

翻译：近年来，几何深度学习领域涌现出多种针对黎曼流形上数据的神经网络。这类网络在具有层次结构的图数据学习或自然科学中流形值数据学习方面具有重要价值。这些网络通常受标准欧氏神经网络启发并直接对其泛化，然而实现这种扩展颇具挑战性，目前仅对少数特定流形成功应用。本研究聚焦残差神经网络（ResNet），提出一种以几何原理为准则的通用黎曼流形扩展方法。ResNet最初为解决梯度消失问题而设计，因其优异的学习性能、卓越的实验效果及构建多样化网络时的易集成性，已成为机器学习领域的通用架构。实验表明，我们的黎曼ResNet继承了这些优良特性：在与现有面向双曲空间及对称正定矩阵流形设计的流形神经网络对比时，本方法在相关测试指标和训练动态上均实现性能超越。