In this work, a comprehensive numerical study involving analysis and experiments shows why a two-layer neural network has difficulties handling high frequencies in approximation and learning when machine precision and computation cost are important factors in real practice. In particular, the following fundamental computational issues are investigated: (1) the best accuracy one can achieve given a finite machine precision, (2) the computation cost to achieve a given accuracy, and (3) stability with respect to perturbations. The key to the study is the spectral analysis of the corresponding Gram matrix of the activation functions which also shows how the properties of the activation function play a role in the picture.

翻译：本研究通过综合数值分析与实验，揭示了当机器精度与计算成本成为实际应用中的关键因素时，两层神经网络在处理高频信号的逼近与学习任务时为何存在困难。特别地，本文探讨了以下基础计算问题：（1）在有限机器精度下可实现的最佳精度；（2）为达到给定精度所需的计算成本；以及（3）对扰动的稳定性。本研究的核心在于对激活函数对应Gram矩阵的谱分析，该分析同时揭示了激活函数特性在此过程中的作用机制。