Mode connectivity is a phenomenon where trained models are connected by a path of low loss. We reframe this in the context of Information Geometry, where neural networks are studied as spaces of parameterized distributions with curved geometry. We hypothesize that shortest paths in these spaces, known as geodesics, correspond to mode-connecting paths in the loss landscape. We propose an algorithm to approximate geodesics and demonstrate that they achieve mode connectivity.

翻译：模式连通性是指训练完成的模型之间存在一条低损失路径相连的现象。本文从信息几何的角度重新审视这一现象，其中神经网络被研究为具有弯曲几何结构的参数化分布空间。我们假设这些空间中的最短路径（即测地线）对应于损失景观中的模式连接路径。我们提出了一种近似测地线的算法，并证明该算法能够实现模式连通性。