We study the complexity of sampling from the stationary distribution of a mean-field SDE, or equivalently, the complexity of minimizing a functional over the space of probability measures which includes an interaction term. Our main insight is to decouple the two key aspects of this problem: (1) approximation of the mean-field SDE via a finite-particle system, via uniform-in-time propagation of chaos, and (2) sampling from the finite-particle stationary distribution, via standard log-concave samplers. Our approach is conceptually simpler and its flexibility allows for incorporating the state-of-the-art for both algorithms and theory. This leads to improved guarantees in numerous settings, including better guarantees for optimizing certain two-layer neural networks in the mean-field regime. A key technical contribution is to establish a new uniform-in-$N$ log-Sobolev inequality for the stationary distribution of the mean-field Langevin dynamics.
翻译:本文研究了从平均场随机微分方程(SDE)的平稳分布中采样的复杂度问题,等价地,即研究在包含交互项的概率测度空间上最小化某个泛函的复杂度。我们的核心思路是将该问题的两个关键方面解耦:(1)通过一致时间传播混沌,用有限粒子系统逼近平均场SDE;(2)利用标准的对数凹采样器,从有限粒子系统的平稳分布中采样。我们的方法在概念上更为简洁,其灵活性允许融合算法与理论领域的最新进展。这为众多场景带来了改进的理论保证,包括在平均场机制下优化某些两层神经网络时获得了更优的保证。一项关键的技术贡献是,我们为平均场朗之万动力学的平稳分布建立了一个新的关于粒子数 $N$ 一致的对数索伯列夫不等式。