We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and layers. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes. They are a smooth analogue of polytopes, exhibiting geometric duality. This finding initiates research of discrete geometry via machine learning and also a visualization of trained networks.

翻译：我们发现，采用ReLU激活函数的简单神经网络能够在不同维度下生成多面体，以此作为单位球面的近似。多面体的类型受网络结构（如单元数量和层数）调控。对于多种激活函数，我们获得了多面体的泛化形式，并将其称为神经多面体。它们是多面体的光滑对应物，展现出几何对偶性。这一发现开启了通过机器学习研究离散几何的途径，同时也提供了对训练后网络的可视化方法。