One of the main challenges in modern deep learning is to understand why such over-parameterized models perform so well when trained on finite data. A way to analyze this generalization concept is through the properties of the associated loss landscape. In this work, we consider the loss landscape as an embedded Riemannian manifold and show that the differential geometric properties of the manifold can be used when analyzing the generalization abilities of a deep net. In particular, we focus on the scalar curvature, which can be computed analytically for our manifold, and show connections to several settings that potentially imply generalization.

翻译：现代深度学习面临的主要挑战之一是理解为何这种过参数化模型在有限数据上训练时表现如此出色。分析这一泛化概念的一种方法是通过关联损失景观的性质。本文将损失景观视为嵌入的黎曼流形，并展示该流形的微分几何性质可用于分析深度网络的泛化能力。特别地，我们重点研究标量曲率（它可针对我们的流形进行解析计算），并展示其与若干可能隐含泛化性的情境之间的联系。