Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear. We address this gap by developing a quantitative framework in which depth admits a precise scale-dependent interpretation. Specifically, we design a single shared mixed-activation architecture of fixed width $2dN+d+2$ and any prescribed finite depth such that each intermediate readout $Φ_\ell$ is itself an approximant to the target function $f$. For $f\in L^p([0,1]^d)$ with $p\in [1,\infty)$, the approximation error of $Φ_\ell$ is controlled by $(2d+1)$ times the $L^p$ modulus of continuity at the geometric scale $N^{-\ell}$ for all $\ell$. The estimate reduces to the geometric rate $(2d+1)N^{-\ell}$ if $f$ is $1$-Lipschitz. Our network design is inspired by multigrade deep learning, where depth serves as a progressive refinement mechanism: each new correction targets residual information at a finer scale while the earlier correction terms remain part of the later readouts, yielding a nested architecture that supports adaptive refinement without redesigning the preceding network.

翻译：深度普遍被视为深度神经网络成功的关键因素，而标准神经网络逼近理论通常仅针对最终输出提供理论保证，对中间层的作用大多未予阐明。我们通过构建一个定量框架来填补这一空白，在该框架中深度具有精确的尺度依赖解释。具体而言，我们设计了一种共享混合激活的单一架构，其固定宽度为$2dN+d+2$，且具有任意预设的有限深度，使得每个中间读出层$Φ_\ell$本身都是目标函数$f$的逼近器。对于$p\in [1,\infty)$的$f\in L^p([0,1]^d)$，$Φ_\ell$的逼近误差由$(2d+1)$倍$L^p$模在几何尺度$N^{-\ell}$上的连续模控制，该控制对所有$\ell$成立。若$f$为$1$-Lipschitz函数，则该估计退化为几何速率$(2d+1)N^{-\ell}$。我们的网络设计受多级深度学习启发，其中深度作为渐进式精化机制：每次新修正以更精细尺度上的残差信息为目标，同时先前修正项保留在后续读出层中，从而形成支持自适应精化的嵌套架构，且无需重新设计前置网络。