We analyze the universal approximation constraints of narrow Residual Neural Networks (ResNets) both theoretically and numerically. For deep neural networks without input space augmentation, a central constraint is the inability to represent critical points of the input-output map. We prove that this has global consequences for target function approximations and show that the manifestation of this defect is typically a shift of the critical point to infinity, which we call the ``tunnel effect'' in the context of classification tasks. While ResNets offer greater expressivity than standard multilayer perceptrons (MLPs), their capability strongly depends on the signal ratio between the skip and residual channels. We establish quantitative approximation bounds for both the residual-dominant (close to MLP) and skip-dominant (close to neural ODE) regimes. These estimates depend explicitly on the channel ratio and uniform network weight bounds. Low-dimensional examples further provide a detailed analysis of the different ResNet regimes and how architecture-target incompatibility influences the approximation error.

翻译：我们从理论和数值两方面分析了窄残差神经网络（ResNets）的通用逼近约束。对于未进行输入空间增广的深度神经网络，其核心约束在于无法表示输入-输出映射的临界点。我们证明，这一缺陷会对目标函数逼近产生全局性影响，并表明该缺陷的典型表现是临界点偏移至无穷远处——在分类任务背景下，我们将其称为"隧道效应"。尽管ResNets比标准多层感知机（MLPs）具有更强的表达能力，但其能力很大程度上依赖于跳跃连接与残差通道之间的信号比率。我们为残差主导（接近MLP）和跳跃主导（接近神经常微分方程）两种机制建立了定量逼近界限。这些估计显式依赖于通道比率和网络权重的统一界。通过低维示例，我们进一步详细分析了不同ResNet机制，以及架构与目标不兼容性如何影响逼近误差。