We use elliptic partial differential equations (PDEs) as examples to show various properties and behaviors when shallow neural networks (SNNs) are used to represent the solutions. In particular, we study the numerical ill-conditioning, frequency bias, and the balance between the differential operator and the shallow network representation for different formulations of the PDEs and with various activation functions. Our study shows that the performance of Physics-Informed Neural Networks (PINNs) or Deep Ritz Method (DRM) using linear SNNs with power ReLU activation is dominated by their inherent ill-conditioning and spectral bias against high frequencies. Although this can be alleviated by using non-homogeneous activation functions with proper scaling, achieving such adaptivity for nonlinear SNNs remains costly due to ill-conditioning.

翻译：我们以椭圆型偏微分方程（PDEs）为例，展示了使用浅层神经网络（SNNs）表示解时的多种性质与行为。具体而言，我们研究了不同PDE表述形式及多种激活函数下数值病态性、频率偏差以及微分算子与浅层网络表示之间的平衡关系。研究表明，采用幂ReLU激活的线性SNNs构建的物理信息神经网络（PINNs）或深度Ritz方法（DRM）的性能主要受其固有的病态性和对高频成分的谱偏差所主导。虽然通过使用具有适当缩放的非齐次激活函数可以缓解此问题，但由于病态性影响，为非线性SNNs实现此类自适应仍代价高昂。