We study the expressivity of ReLU neural networks in the setting of a binary classification problem from a topological perspective. Recently, empirical studies showed that neural networks operate by changing topology, transforming a topologically complicated data set into a topologically simpler one as it passes through the layers. This topological simplification has been measured by Betti numbers, which are algebraic invariants of a topological space. We use the same measure to establish lower and upper bounds on the topological simplification a ReLU neural network can achieve with a given architecture. We therefore contribute to a better understanding of the expressivity of ReLU neural networks in the context of binary classification problems by shedding light on their ability to capture the underlying topological structure of the data. In particular the results show that deep ReLU neural networks are exponentially more powerful than shallow ones in terms of topological simplification. This provides a mathematically rigorous explanation why deeper networks are better equipped to handle complex and topologically rich data sets.

翻译：我们从拓扑学的角度研究ReLU神经网络在二分类问题中的表达能力。近期实证研究表明，神经网络通过改变拓扑结构来运作，在数据逐层传递过程中将拓扑复杂的数据集转化为拓扑更简单的形式。这种拓扑简化可通过贝蒂数进行度量，该指标是拓扑空间的代数不变量。我们使用相同度量方法，针对给定架构的ReLU神经网络能够实现的拓扑简化程度建立了上下界。通过揭示神经网络捕捉数据底层拓扑结构的能力，我们的研究为理解ReLU神经网络在二分类问题中的表达能力提供了新视角。特别值得注意的是，研究结果表明：在拓扑简化能力方面，深层ReLU神经网络相较浅层网络具有指数级优势。这为"深层网络为何能更有效处理复杂且拓扑丰富的数据集"提供了严格的数学解释。