The study of universal approximation properties (UAP) for neural networks (NN) has a long history. When the network width is unlimited, only a single hidden layer is sufficient for UAP. In contrast, when the depth is unlimited, the width for UAP needs to be not less than the critical width $w^*_{\min}=\max(d_x,d_y)$, where $d_x$ and $d_y$ are the dimensions of the input and output, respectively. Recently, \cite{cai2022achieve} shows that a leaky-ReLU NN with this critical width can achieve UAP for $L^p$ functions on a compact domain $K$, \emph{i.e.,} the UAP for $L^p(K,\mathbb{R}^{d_y})$. This paper examines a uniform UAP for the function class $C(K,\mathbb{R}^{d_y})$ and gives the exact minimum width of the leaky-ReLU NN as $w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1:2d_x}$, which involves the effects of the output dimensions. To obtain this result, we propose a novel lift-flow-discretization approach that shows that the uniform UAP has a deep connection with topological theory.

翻译：神经网络通用逼近性质（UAP）的研究历史悠久。当网络宽度无限制时，仅需单个隐藏层即可实现UAP；反之，当深度无限制时，实现UAP所需宽度不得小于临界宽度$w^*_{\min}=\max(d_x,d_y)$，其中$d_x$和$d_y$分别为输入与输出维度。近期，文献\cite{cai2022achieve}表明，具有此临界宽度的Leaky-ReLU神经网络可在紧致域$K$上实现对$L^p$函数的UAP，即$L^p(K,\mathbb{R}^{d_y})$空间的UAP。本文针对函数类$C(K,\mathbb{R}^{d_y})$研究均匀UAP，并给出Leaky-ReLU神经网络精确最小宽度为$w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1:2d_x}$，该结果体现了输出维度的影响。为获得此结论，我们提出一种新颖的升流-离散化方法，证明均匀UAP与拓扑理论存在深层联系。