We present an estimation of the condition numbers of the \emph{mass} and \emph{stiffness} matrices arising from shallow ReLU$^k$ neural networks defined on the unit sphere~$\mathbb{S}^d$. In particular, when $\{\theta_j^*\}_{j=1}^n \subset \mathbb{S}^d$ is \emph{antipodally quasi-uniform}, the condition number is sharp. Indeed, in this case, we obtain sharp asymptotic estimates for the full spectrum of eigenvalues and characterize the structure of the corresponding eigenspaces, showing that the smallest eigenvalues are associated with an eigenbasis of low-degree polynomials while the largest eigenvalues are linked to high-degree polynomials. This spectral analysis establishes a precise correspondence between the approximation power of the network and its numerical stability.
翻译:本文针对定义在单位球面~$\mathbb{S}^d$上的浅层ReLU$^k$神经网络,给出了其质量矩阵与刚度矩阵条件数的估计。特别地,当节点集$\{\theta_j^*\}_{j=1}^n \subset \mathbb{S}^d$满足对径拟均匀分布时,条件数估计达到最优。在此情形下,我们获得了特征值全谱的渐近最优估计,并刻画了对应特征空间的结构:最小特征值对应于低阶多项式构成的特征基,而最大特征值则与高阶多项式相关联。该谱分析结果在网络的逼近能力与数值稳定性之间建立了精确的对应关系。