There is a large variety of machine learning methodologies that are based on the extraction of spectral geometric information from data. However, the implementations of many of these methods often depend on traditional eigensolvers, which present limitations when applied in practical online big data scenarios. To address some of these challenges, researchers have proposed different strategies for training neural networks as alternatives to traditional eigensolvers, with one such approach known as Spectral Neural Network (SNN). In this paper, we investigate key theoretical aspects of SNN. First, we present quantitative insights into the tradeoff between the number of neurons and the amount of spectral geometric information a neural network learns. Second, we initiate a theoretical exploration of the optimization landscape of SNN's objective to shed light on the training dynamics of SNN. Unlike typical studies of convergence to global solutions of NN training dynamics, SNN presents an additional complexity due to its non-convex ambient loss function.

翻译：基于数据中谱几何信息提取的机器学习方法种类繁多。然而，这些方法中的许多实现往往依赖于传统的特征求解器，这在实际在线大数据场景中应用时存在局限性。为了应对这些挑战，研究者提出了不同的策略来训练神经网络作为传统特征求解器的替代方案，其中一种方法称为谱神经网络（SNN）。本文研究了SNN的关键理论方面。首先，我们定量分析了神经元数量与神经网络学习到的谱几何信息量之间的权衡关系。其次，我们初步探索了SNN目标的优化景观，以揭示SNN的训练动态。与典型的神经网络训练动态全局解收敛研究不同，SNN由于其非凸的全局损失函数而呈现出额外的复杂性。