Deep neural networks (DNNs) have achieved remarkable success in scientific computing, yet they often suffer from spectral bias in capturing oscillatory and multiscale behaviors. In this study, we investigate this limitation by examining the failure of shallow ReLU neural networks in fitting high-frequency functions. This observation identifies two important factors in resolving rapid oscillations: the initial slope scale and the distribution of partition points induced by the networks. Motivated by this analysis, we propose RepNet, a reparameterized DNN model for ReLU and tanh networks designed for high-frequency and multiscale problems. The key idea is to reparameterize the weights and biases in the first hidden layer, which enables effective control of the initial slope scale and provides an appropriate distribution of the initial partition points. Furthermore, treating the reparameterized weights and biases as trainable parameters allows the DNN to achieve adaptive frequency scaling during training. In addition, we derive quantitative estimates for the output and slope magnitudes of the reparameterized DNN to guide the initialization of the proposed method. Numerical experiments, including multiscale one- and four-dimensional function approximation, forward and inverse PDE problems in combination with physics-informed neural networks (PINNs), and operator learning, demonstrate that RepNet improves the predicted accuracy of vanilla DNNs in capturing highly oscillatory features with slightly additional computational cost. These results indicate that RepNet provides an effective and flexible approach for overcoming spectral bias and applying DNNs to multiscale problems.

翻译：深度神经网络（DNNs）在科学计算领域取得了显著成功，但在捕捉振荡和多尺度行为时往往存在谱偏差问题。本研究通过考察浅层ReLU神经网络在拟合高频函数时的失效机制，探索了这一局限性。该观察揭示了解决快速振荡问题的两个重要因素：初始斜率尺度与网络诱导的分割点分布。基于此分析，我们提出RepNet——一种针对高频及多尺度问题设计的、适用于ReLU和tanh网络的重参数化DNN模型。其核心思想是对第一隐藏层的权重与偏置进行重参数化，从而有效控制初始斜率尺度，并提供恰当的初始分割点分布。进一步地，将重参数化后的权重与偏置视为可训练参数，使得DNN能在训练过程中实现自适应频率缩放。此外，我们推导了重参数化DNN输出与斜率幅度的定量估计，以指导所提方法的初始化。数值实验涵盖多尺度一维与四维函数逼近、基于物理信息神经网络（PINNs）的正反问题求解以及算子学习，结果表明RepNet能以微小的额外计算成本提升原始DNN在捕捉高度振荡特征时的预测精度。这些结果证明RepNet为克服谱偏差并将DNN应用于多尺度问题提供了一种有效且灵活的方法。