These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training.
翻译:本讲座于2022年Les Houches统计物理与机器学习暑期学校授课,聚焦深度神经网络的无限宽度极限与大宽度机制。涵盖内容包括这些网络的多种统计与动态特性。具体而言,讲座者讨论了随机深度神经网络的性质;训练后的深度神经网络与线性模型、核方法及高斯过程在无限宽度极限下的联系;以及针对宽度较大但仍有限的网络,在初始化和训练后的微扰与非微扰处理方法。