We study the expressivity of ReLU neural networks in the setting of a binary classification problem from a topological perspective. Recently, empirical studies showed that neural networks operate by changing topology, transforming a topologically complicated data set into a topologically simpler one as it passes through the layers. This topological simplification has been measured by Betti numbers, which are algebraic invariants of a topological space. We use the same measure to establish lower and upper bounds on the topological simplification a ReLU neural network can achieve with a given architecture. We therefore contribute to a better understanding of the expressivity of ReLU neural networks in the context of binary classification problems by shedding light on their ability to capture the underlying topological structure of the data. In particular the results show that deep ReLU neural networks are exponentially more powerful than shallow ones in terms of topological simplification. This provides a mathematically rigorous explanation why deeper networks are better equipped to handle complex and topologically rich datasets.

翻译：我们从拓扑视角研究了二分类问题中ReLU神经网络的表达性。近期实证研究表明，神经网络通过改变拓扑结构运作——当数据逐层传递时，会将拓扑复杂的数据集转化为拓扑简单的形式。这种拓扑简化可通过贝蒂数（拓扑空间的代数不变量）进行度量。我们采用相同度量，为给定架构的ReLU神经网络所能实现的拓扑简化建立了上下界。通过揭示网络捕捉数据底层拓扑结构的能力，我们进一步深化了对二分类问题中ReLU神经网络表达性的理解。研究结果特别表明，在拓扑简化能力方面，深层ReLU神经网络的指数级优势远超浅层网络。这为"深度网络更擅长处理复杂且拓扑丰富的数据集"这一现象提供了严谨的数学解释。