We study approximation by shallow ReLU$^s$ networks, $σ_s(t)=\max{0,t}^s$, and the generalization behavior of such networks under $\ell_1$ path-norm control. For the $L^p$-type integral spaces $\widetilde{\mathcal{F}}_{p,τ_d,s}$, $1\le p\le2$, we establish approximation bounds for shallow networks using spherical harmonic analysis. In particular, when the parameter measure is the uniform measure $τ_d$ and $p<p^*=(2d+2)/(d+3)$, we obtain the rate $O(m^{-1/2-d(2-p)/(2d(2-p)+2p(2s+d+1))}\log^{3/2}m)$, which improves the corresponding random-feature rate. We also derive approximation rates for Sobolev spaces $W^{α,p}$ in the range $1\le p<2$ by embedding them into spectral Barron spaces. Finally, for nonparametric regression with sub-Gaussian noise, we prove minimax-optimal generalization bounds for path-norm-regularized shallow ReLU$^s$ networks over Barron and Sobolev spaces, with matching lower bounds up to logarithmic factors.
翻译:本文研究由ReLU$^s$网络(激活函数为$\sigma_s(t)=\max\{0,t\}^s$)构成的浅层网络的逼近性质,以及在$\ell_1$路径范数约束下的泛化行为。针对$L^p$型积分空间$\widetilde{\mathcal{F}}_{p,\tau_d,s}$($1\le p\le2$),我们利用球谐分析建立了浅层网络的逼近界。特别地,当参数测度为均匀测度$\tau_d$且$p<p^*=(2d+2)/(d+3)$时,得到了逼近速率$O(m^{-1/2-d(2-p)/(2d(2-p)+2p(2s+d+1))}\log^{3/2}m)$,该结果优于相应的随机特征速率。进一步,通过将Sobolev空间$W^{\alpha,p}$($1\le p<2$)嵌入谱Barron空间,我们推导了其逼近速率。最后,针对次高斯噪声下的非参数回归问题,我们证明了在Barron空间与Sobolev空间上,经路径范数正则化的浅层ReLU$^s$网络可达到极小极大最优的泛化界,且下界与上界在对数因子意义下匹配。