The architecture of a neural network and the choice of its activation function are both fundamental to its performance. Equally important is ensuring that these two elements are well matched, as their alignment is key to effective representation and learning. In this paper, we introduce the Fourier Multi-Component and Multi-Layer Neural Network (FMMNN), a model that combines sine-type activations with the multi-component and multi-layer structure of MMNNs. In an FMMNN, each component is represented as a trainable linear combination of fixed random sine-type basis functions, while multi-layer composition generates more complex and adaptive high-frequency features. We establish that FMMNNs retain exponential expressive power for function approximation even under a low-rank architectural structure. We also analyze the optimization landscape of FMMNNs and find it to be substantially more favorable than that of standard fully connected neural networks, especially for high-frequency targets. In addition, we propose a scaled random initialization method for the first-layer weights in FMMNNs, which accelerates training and improves final performance when sufficient samples are available. Extensive numerical experiments support our theoretical insights, showing that FMMNNs achieve strong accuracy and favorable convergence behavior on oscillatory function-approximation benchmarks.

翻译：神经网络的架构设计及其激活函数的选择对其性能至关重要。确保这两个要素良好匹配同样关键，因为它们的协同对齐是实现有效表示和学习的基础。本文引入了傅里叶多分量与多层神经网络（FMMNN），该模型将正弦型激活函数与MMNN的多分量、多层结构相结合。在FMMNN中，每个分量被表示为固定的随机正弦型基函数的可训练线性组合，而多层组合则能生成更复杂且更具适应性的高频特征。我们证明，即使采用低秩架构，FMMNN在函数逼近中仍能保持指数级的表达能力。我们还分析了FMMNN的优化景观，发现其相比标准全连接神经网络具有显著优势，尤其是在处理高频目标时。此外，我们提出了一种针对FMMNN第一层权重的缩放随机初始化方法，该方法在样本充足时能加速训练并提升最终性能。大量数值实验支持了我们的理论分析，结果表明FMMNN在对振荡函数逼近的基准测试中实现了高精度且具备有利的收敛行为。