We study Langevin dynamics with noise projected onto the directions orthogonal to an isometric group action. This mathematical model is introduced to shed new light on the effects of symmetry on stochastic gradient descent for over-parametrized models. Our main result identifies a novel form of implicit regularization: when the initial and target density are both invariant under the group action, Langevin dynamics with projected noise is equivalent in law to Langevin dynamics with isotropic diffusion but with an additional drift term proportional to the negative log volume of the group orbit. We prove this result by constructing a coupling of the two processes via a third process on the group itself, and identify the additional drift as the mean curvature of the orbits.

翻译：本研究探讨了噪声投影到等距群作用正交方向上的朗格文动力学。引入该数学模型旨在为过参数化模型中随机梯度下降的对称性效应提供新的理论视角。我们的主要成果揭示了一种新颖的隐式正则化形式：当初始密度与目标密度均在群作用下保持不变时，带投影噪声的朗格文动力学在分布律上等价于各向同性扩散的朗格文动力学，但需附加一个与群轨道负对数体积成正比的漂移项。我们通过在群自身上构建第三过程来实现两个过程的耦合证明，并识别出该附加漂移项即为轨道平均曲率的数学表征。