In many applications of machine learning, a large number of variables are considered. Motivated by machine learning of interacting particle systems, we consider the situation when the number of input variables goes to infinity. First, we continue the recent investigation of the mean field limit of kernels and their reproducing kernel Hilbert spaces, completing the existing theory. Next, we provide results relevant for approximation with such kernels in the mean field limit, including a representer theorem. Finally, we use these kernels in the context of statistical learning in the mean field limit, focusing on Support Vector Machines. In particular, we show mean field convergence of empirical and infinite-sample solutions as well as the convergence of the corresponding risks. On the one hand, our results establish rigorous mean field limits in the context of kernel methods, providing new theoretical tools and insights for large-scale problems. On the other hand, our setting corresponds to a new form of limit of learning problems, which seems to have not been investigated yet in the statistical learning theory literature.
翻译:在许多机器学习应用中,需要考虑大量变量。受相互作用粒子系统机器学习的启发,我们考虑输入变量数量趋于无穷的情况。首先,我们延续了近期对核函数及其再生核希尔伯特空间均场极限的研究,完善了现有理论。其次,我们提供了在均场极限下利用此类核函数进行逼近的相关结果,包括一个表征定理。最后,我们将这些核函数应用于均场极限下的统计学习场景,重点聚焦支持向量机。特别地,我们证明了经验解与无限样本解的均场收敛性以及相应风险的收敛性。一方面,我们的结果为核方法建立了严格的均场极限,为大规模问题提供了新的理论工具与见解。另一方面,我们的设定对应于一种新型学习问题极限形式,这在统计学习理论文献中似乎尚未被研究。