Neural networks activated by the rectified linear unit (ReLU) play a central role in the recent development of deep learning. The topic of approximating functions from H\"older spaces by these networks is crucial for understanding the efficiency of the induced learning algorithms. Although the topic has been well investigated in the setting of deep neural networks with many layers of hidden neurons, it is still open for shallow networks having only one hidden layer. In this paper, we provide rates of uniform approximation by these networks. We show that ReLU shallow neural networks with $m$ hidden neurons can uniformly approximate functions from the H\"older space $W_\infty^r([-1, 1]^d)$ with rates $O((\log m)^{\frac{1}{2} +d}m^{-\frac{r}{d}\frac{d+2}{d+4}})$ when $r<d/2 +2$. Such rates are very close to the optimal one $O(m^{-\frac{r}{d}})$ in the sense that $\frac{d+2}{d+4}$ is close to $1$, when the dimension $d$ is large.

翻译：以修正线性单元（ReLU）为激活函数的神经网络在深度学习的最新发展中发挥着核心作用。利用这些网络逼近Hölder空间中的函数这一课题，对于理解由此产生的学习算法的效率至关重要。尽管该课题在具有多层隐藏神经元的深度神经网络背景下已得到充分研究，但对于仅有一个隐藏层的浅层网络而言，相关问题仍未解决。在本文中，我们给出了此类网络的均匀逼近速率。我们证明，当$r<d/2+2$时，具有$m$个隐藏神经元的ReLU浅层神经网络能够以$O((\log m)^{\frac{1}{2}+d}m^{-\frac{r}{d}\frac{d+2}{d+4}})$的速率均匀逼近Hölder空间$W_\infty^r([-1,1]^d)$中的函数。当维度$d$较大时，该速率在$\frac{d+2}{d+4}$接近$1$的意义下，非常接近最优速率$O(m^{-\frac{r}{d}})$。