We present experimental results highlighting two key differences resulting from the choice of training algorithm for two-layer neural networks. The spectral bias of neural networks is well known, while the spectral bias dependence on the choice of training algorithm is less studied. Our experiments demonstrate that an adaptive random Fourier features algorithm (ARFF) can yield a spectral bias closer to zero compared to the stochastic gradient descent optimizer (SGD). Additionally, we train two identically structured classifiers, employing SGD and ARFF, to the same accuracy levels and empirically assess their robustness against adversarial noise attacks.

翻译：我们通过实验揭示了两种训练算法在两层神经网络中产生的两个关键差异。神经网络的光谱偏差是广为人知的现象，但训练算法选择对其依赖性的研究尚不充分。我们的实验表明，与随机梯度下降优化器（SGD）相比，自适应随机傅里叶特征算法（ARFF）能够使光谱偏差更趋近于零。此外，我们分别使用SGD和ARFF训练了两个结构相同的分类器，使其达到相同的准确率水平，并通过实证评估了它们在面对对抗性噪声攻击时的鲁棒性。