This paper investigates the relationship between the universal approximation property of deep neural networks and topological characteristics of datasets. Our primary contribution is to introduce data topology-dependent upper bounds on the network width. Specifically, we first show that a three-layer neural network, applying a ReLU activation function and max pooling, can be designed to approximate an indicator function over a compact set, one that is encompassed by a tight convex polytope. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. Further, we calculate upper bounds in relation to the Betti numbers of select topological spaces. Finally, we prove the universal approximation property of three-layer ReLU networks using our topological approach. We also verify that gradient descent converges to the network structure proposed in our study.

翻译：本文研究了深度神经网络的通用逼近性质与数据集拓扑特征之间的关系。我们的主要贡献是引入了依赖于数据拓扑的网络宽度上界。具体而言，我们首先证明了采用ReLU激活函数和最大池化的三层神经网络能够近似定义在紧集上的指示函数，且该紧集被一个紧凸多面体所包含。随后，我们将此结果推广至单纯复形，推导出基于其拓扑结构的宽度上界。进一步地，我们计算了与特定拓扑空间贝蒂数相关的上界。最后，利用我们的拓扑方法证明了三层ReLU网络的通用逼近性质，并验证了梯度下降收敛至本研究提出的网络结构。