Neural networks can be thought of as applying a transformation to an input dataset. The way in which they change the topology of such a dataset often holds practical significance for many tasks, particularly those demanding non-homeomorphic mappings for optimal solutions, such as classification problems. In this work, we leverage the fact that neural networks are equivalent to continuous piecewise-affine maps, whose rank can be used to pinpoint regions in the input space that undergo non-homeomorphic transformations, leading to alterations in the topological structure of the input dataset. Our approach enables us to make use of the relative homology sequence, with which one can study the homology groups of the quotient of a manifold $\mathcal{M}$ and a subset $A$, assuming some minimal properties on these spaces. As a proof of principle, we empirically investigate the presence of low-rank (topology-changing) affine maps as a function of network width and mean weight. We show that in randomly initialized narrow networks, there will be regions in which the (co)homology groups of a data manifold can change. As the width increases, the homology groups of the input manifold become more likely to be preserved. We end this part of our work by constructing highly non-random wide networks that do not have this property and relating this non-random regime to Dale's principle, which is a defining characteristic of biological neural networks. Finally, we study simple feedforward networks trained on MNIST, as well as on toy classification and regression tasks, and show that networks manipulate the topology of data differently depending on the continuity of the task they are trained on.

翻译：神经网络可视为对输入数据集施加变换的过程。此类变换改变数据拓扑结构的方式，在实际任务中常具有重要应用价值，尤其对需要非同胚映射以实现最优解的分类问题。本文利用神经网络等价于连续分段仿射映射的特性，基于其秩可定位输入空间中发生非同胚变换的区域，进而导致输入数据拓扑结构的改变。该方法使我们能够利用相对同调序列，在满足空间最小性质的前提下，研究流形$\mathcal{M}$与子集$A$商空间的同调群。作为原理验证，我们通过实验探究了低秩（拓扑改变）仿射映射随网络宽度和平均权重变化的规律，证明在随机初始化的窄网络中，数据流形的（上）同调群可能在某些区域发生变化。随着网络宽度增加，输入流形的同调群更可能被完整保留。本工作最后构建了不具此特性的高度非随机宽网络，并将其与生物神经网络的核心特征——戴尔原则建立关联。我们在MNIST数据集及玩具分类/回归任务中训练简单前馈网络，结果表明网络对数据拓扑的操作方式取决于训练任务的连续性特征。