In this work, we will establish that the Langevin Monte-Carlo algorithm can learn depth-2 neural nets of any size and for any data and we give non-asymptotic convergence rates for it. We achieve this via showing that under Total Variation distance and q-Renyi divergence, the iterates of Langevin Monte Carlo converge to the Gibbs distribution of Frobenius norm regularized losses for any of these nets, when using smooth activations and in both classification and regression settings. Most critically, the amount of regularization needed for our results is independent of the size of the net. The key observation of ours is that two layer neural loss functions can always be regularized by a constant amount such that they satisfy the Villani conditions, and thus their Gibbs measures satisfy a Poincare inequality.

翻译：本文中，我们将证明 Langevin Monte-Carlo 算法能够学习任意规模、适用于任意数据的深度二层神经网络，并给出其非渐近收敛速率。我们通过证明在总变差距离和 q-Renyi 散度下，当使用光滑激活函数且在分类与回归两种设定中，对于任意此类网络，Langevin Monte-Carlo 的迭代序列均收敛于 Frobenius 范数正则化损失的 Gibbs 分布。最关键的是，我们结果所需的正则化量与网络规模无关。我们的核心观察是：二层神经损失函数总可通过一个常数量的正则化，使其满足 Villani 条件，从而其 Gibbs 测度满足 Poincaré 不等式。