In this work, we will establish that the Langevin Monte-Carlo algorithm can learn depth-2 neural nets of any size and for any data and we give non-asymptotic convergence rates for it. We achieve this via showing that under Total Variation distance and q-Renyi divergence, the iterates of Langevin Monte Carlo converge to the Gibbs distribution of Frobenius norm regularized losses for any of these nets, when using smooth activations and in both classification and regression settings. Most critically, the amount of regularization needed for our results is independent of the size of the net. This result combines several recent observations, like our previous papers showing that two-layer neural loss functions can always be regularized by a certain constant amount such that they satisfy the Villani conditions, and thus their Gibbs measures satisfy a Poincare inequality.

翻译：本文中，我们将证明 Langevin Monte-Carlo 算法能够学习任意规模、适用于任意数据的深度二神经网络，并给出其非渐近收敛速率。我们通过证明在总变差距离和 q-Renyi 散度下，当使用平滑激活函数且在分类与回归两种设定中，Langevin Monte Carlo 的迭代序列会收敛到这些网络经 Frobenius 范数正则化损失所对应的 Gibbs 分布，从而达成这一结论。最关键的是，我们结果所需的正则化量与网络规模无关。该结果综合了若干近期发现，例如我们先前论文中表明：双层神经损失函数总可通过某个固定量进行正则化，使其满足 Villani 条件，因而其 Gibbs 测度满足 Poincaré 不等式。