Statistical physics provides tools for analyzing high-dimensional problems in machine learning and theoretical neuroscience. These calculations, particularly those using the replica method, often involve lengthy derivations that can obscure physical interpretation. We give concise, non-replica derivations of several key results and highlight their underlying similarities. Specifically, we introduce a cavity approach to analyzing high-dimensional learning problems and apply it to three cases: perceptron classification of points, perceptron classification of manifolds, and kernel ridge regression. These problems share a common structure -- a bipartite system of interacting feature and datum variables -- enabling a unified analysis. For perceptron-capacity problems, we identify a symmetry that allows derivation of correct capacities through a na\"ive method. These results match those obtained through the replica method.
翻译:统计物理学为机器学习和理论神经科学中的高维问题分析提供了工具。这些计算,特别是使用复本方法进行的计算,通常涉及冗长的推导过程,可能掩盖物理本质。我们给出了几个关键结果的简洁非复本推导,并强调了它们的内在相似性。具体而言,我们引入了一种分析高维学习问题的空腔方法,并将其应用于三种情况:点的感知机分类、流形的感知机分类以及核岭回归。这些问题具有共同的结构——一个由相互作用的特征变量和数据变量组成的二部系统——从而实现了统一分析。对于感知机容量问题,我们识别出一种对称性,使得可以通过一种朴素方法推导出正确的容量。这些结果与通过复本方法获得的结果一致。