This work focuses on the analysis of fully connected feed forward ReLU neural networks as they approximate a given, smooth function. In contrast to conventionally studied universal approximation properties under increasing architectures, e.g., in terms of width or depth of the networks, we are concerned with the asymptotic growth of the parameters of approximating networks. Such results are of interest, e.g., for error analysis or consistency results for neural network training. The main result of our work is that, for a ReLU architecture with state of the art approximation error, the realizing parameters grow at most polynomially. The obtained rate with respect to a normalized network size is compared to existing results and is shown to be superior in most cases, in particular for high dimensional input.

翻译：本研究聚焦于分析全连接前馈ReLU神经网络在逼近给定光滑函数时的性质。与传统研究网络架构扩展（如增加宽度或深度）下的通用逼近性质不同，我们关注逼近网络参数的渐近增长规律。此类结果对神经网络训练中的误差分析或一致性研究具有重要意义。本工作的主要结论表明：对于具有当前最优逼近误差的ReLU架构，其实现参数至多以多项式速率增长。通过将所得关于归一化网络规模的增长率与现有结果比较，证明该速率在多数情况下（特别是高维输入场景中）具有优越性。