In recent work it has been shown that determining a feedforward ReLU neural network to within high uniform accuracy from point samples suffers from the curse of dimensionality in terms of the number of samples needed. As a consequence, feedforward ReLU neural networks are of limited use for applications where guaranteed high uniform accuracy is required. We consider the question of whether the sampling complexity can be improved by restricting the specific neural network architecture. To this end, we investigate invertible residual neural networks which are foundational architectures in deep learning and are widely employed in models that power modern generative methods. Our main result shows that the residual neural network architecture and invertibility do not help overcome the complexity barriers encountered with simpler feedforward architectures. Specifically, we demonstrate that the computational complexity of approximating invertible residual neural networks from point samples in the uniform norm suffers from the curse of dimensionality. Similar results are established for invertible convolutional Residual neural networks.

翻译：近期研究表明，从点样本中以高均匀精度确定前馈ReLU神经网络所需的样本数量存在维度灾难问题。因此，在需要保证高均匀精度的应用场景中，前馈ReLU神经网络的使用受到限制。本文探讨通过限制特定神经网络架构是否能够改善采样复杂度。为此，我们研究了可逆残差神经网络——该架构是深度学习的基础结构，广泛应用于驱动现代生成方法的模型中。我们的主要结果表明，残差神经网络架构及其可逆特性并不能帮助克服在简单前馈架构中遇到的复杂度障碍。具体而言，我们证明了在均匀范数下通过点样本逼近可逆残差神经网络的计算复杂度同样遭受维度灾难。对于可逆卷积残差神经网络，我们也建立了类似的结果。