The study of universal approximation properties (UAP) for neural networks (NN) has a long history. When the network width is unlimited, only a single hidden layer is sufficient for UAP. In contrast, when the depth is unlimited, the width for UAP needs to be not less than the critical width $w^*_{\min}=\max(d_x,d_y)$, where $d_x$ and $d_y$ are the dimensions of the input and output, respectively. Recently, \cite{cai2022achieve} shows that a leaky-ReLU NN with this critical width can achieve UAP for $L^p$ functions on a compact domain ${K}$, \emph{i.e.,} the UAP for $L^p({K},\mathbb{R}^{d_y})$. This paper examines a uniform UAP for the function class $C({K},\mathbb{R}^{d_y})$ and gives the exact minimum width of the leaky-ReLU NN as $w_{\min}=\max(d_x,d_y)+\Delta (d_x, d_y)$, where $\Delta (d_x, d_y)$ is the additional dimensions for approximating continuous functions with diffeomorphisms via embedding. To obtain this result, we propose a novel lift-flow-discretization approach that shows that the uniform UAP has a deep connection with topological theory.

翻译：神经网络通用逼近性质的研究历史悠久。当网络宽度不受限时，仅需单个隐藏层即可实现通用逼近。相反，当深度不受限时，通用逼近所需宽度不得小于临界宽度$w^*_{\min}=\max(d_x,d_y)$，其中$d_x$和$d_y$分别为输入和输出的维度。近期文献\cite{cai2022achieve}表明，具有此临界宽度的Leaky-ReLU神经网络可在紧致域${K}$上实现$L^p$函数的通用逼近，即对$L^p({K},\mathbb{R}^{d_y})$的通用逼近。本文研究了函数类$C({K},\mathbb{R}^{d_y})$的均匀通用逼近，并给出了Leaky-ReLU神经网络的最小精确宽度$w_{\min}=\max(d_x,d_y)+\Delta (d_x, d_y)$，其中$\Delta (d_x, d_y)$是通过嵌入微分同胚逼近连续函数所需的额外维度。为获得此结果，我们提出了一种新颖的流提升离散化方法，揭示了均匀通用逼近与拓扑理论之间的深层联系。