Let $\Omega = [0,1]^d$ be the unit cube in $\mathbb{R}^d$. We study the problem of how efficiently, in terms of the number of parameters, deep neural networks with the ReLU activation function can approximate functions in the Sobolev space $W^s(L_q(\Omega))$ with error measured in $L_p(\Omega)$. This problem is important when studying the application of neural networks in scientific computing and has previously been completely solved only in the case $p=q=\infty$. Our contribution is to provide a complete solution for all $1\leq p,q\leq \infty$ and $s > 0$, including asymptotically matching upper and lower bounds. The key technical tool is a novel bit-extraction technique which gives an optimal encoding of sparse vectors. This enables us to obtain sharp upper bounds in the non-linear regime where $p > q$. We also provide a novel method for deriving $L_p$-approximation lower bounds based upon VC-dimension when $p < \infty$. Our results show that very deep ReLU networks significantly outperform classical methods of approximation in terms of the number of parameters, but that this comes at the cost of parameters which are not encodable.

翻译：$0,1美元=美元。 我们研究的问题是, 使用RELU 激活功能的深神经网络在参数数量上, 能够以$L_ p(\\\\\\\\\\\\美元) 美元测量出差错的 Sobollev 空间 $W}(L_q (q)(\\\\\\\\\\\\\\\\\美元) $美元。 当研究神经网络在科学计算中的应用时, 这个问题很重要, 并且以前只有在 $p=q=q ⁇ infty$的情况下才完全解决。 我们的贡献是为所有$leq p, qleq\leq\ infty$ 和$ > $0的参数提供一种完整的解决方案, 包括以$L_ pleqleq\\ bety $, restime restitual res of $Lappin restitutional network $L.