The study of universal approximation properties (UAP) for neural networks (NN) has a long history. When the network width is unlimited, only a single hidden layer is sufficient for UAP. In contrast, when the depth is unlimited, the width for UAP needs to be not less than the critical width $w^*_{\min}=\max(d_x,d_y)$, where $d_x$ and $d_y$ are the dimensions of the input and output, respectively. Recently, \cite{cai2022achieve} shows that a leaky-ReLU NN with this critical width can achieve UAP for $L^p$ functions on a compact domain $K$, \emph{i.e.,} the UAP for $L^p(K,\mathbb{R}^{d_y})$. This paper examines a uniform UAP for the function class $C(K,\mathbb{R}^{d_y})$ and gives the exact minimum width of the leaky-ReLU NN as $w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1}$, which involves the effects of the output dimensions. To obtain this result, we propose a novel lift-flow-discretization approach that shows that the uniform UAP has a deep connection with topological theory.

翻译：神经网络通用逼近性质的研究有着悠久历史。当网络宽度不受限时，仅需单隐层即可实现通用逼近；相反，当深度不受限时，实现通用逼近所需宽度不得小于临界宽度$w^*_{\min}=\max(d_x,d_y)$，其中$d_x$和$d_y$分别为输入和输出维度。近期，\cite{cai2022achieve}的研究表明，采用该临界宽度的泄漏ReLU神经网络可在紧致域$K$上实现$L^p$函数类（即$L^p(K,\mathbb{R}^{d_y})$）的通用逼近。本文研究函数类$C(K,\mathbb{R}^{d_y})$的一致通用逼近，并给出泄漏ReLU神经网络的精确最小宽度为$w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1}$，该结果体现了输出维度的影响。为获得此结论，我们提出一种新颖的"提升-流形-离散化"方法，揭示了一致通用逼近与拓扑理论之间的深层联系。