Deep learning algorithms have made incredible strides in the past decade yet due to the complexity of these algorithms, the science of deep learning remains in its early stages. Being an experimentally driven field, it is natural to seek a theory of deep learning within the physics paradigm. As deep learning is largely about learning functions and distributions over functions, statistical field theory, a rich and versatile toolbox for tackling complex distributions over functions (fields) is an obvious choice of formalism. Research efforts carried out in the past few years have demonstrated the ability of field theory to provide useful insights on generalization, implicit bias, and feature learning effects. Here we provide a pedagogical review of this emerging line of research.
翻译:深度学习算法在过去十年取得了令人瞩目的进展,但由于这些算法的复杂性,深度学习科学仍处于早期阶段。作为一个实验驱动的领域,在物理学范式中寻求深度学习理论是自然而然的。由于深度学习主要涉及学习函数及函数上的分布,统计场论作为一个处理函数(场)上复杂分布的丰富而多用途的工具箱,显然是一种合适的形式化选择。过去几年开展的研究工作已证明场论能够为泛化、隐式偏差和特征学习效应提供有价值的见解。本文对这一新兴研究方向进行了教学性综述。