A feedforward neural network using rectified linear units constructs a mapping from inputs to outputs by partitioning its input space into a set of convex regions where points within a region share a single affine transformation. In order to understand how neural networks work, when and why they fail, and how they compare to biological intelligence, we need to understand the organization and formation of these regions. Step one is to design and implement algorithms for exact region enumeration in networks beyond toy examples. In this work, we present parallel algorithms for exact enumeration in deep (and shallow) neural networks. Our work has three main contributions: (1) we present a novel algorithm framework and parallel algorithms for region enumeration; (2) we implement one of our algorithms on a variety of network architectures and experimentally show how the number of regions dictates runtime; and (3) we show, using our algorithm's output, how the dimension of a region's affine transformation impacts further partitioning of the region by deeper layers. To our knowledge, we run our implemented algorithm on networks larger than all of the networks used in the existing region enumeration literature. Further, we experimentally demonstrate the importance of parallelism for region enumeration of any reasonably sized network.

翻译：使用修正线性单元的前馈神经网络通过将输入空间划分为一组凸区域来构建从输入到输出的映射，这些凸区域内的点共享同一个仿射变换。为了理解神经网络的工作机制、何时及为何失效、以及它们与生物智能的对比，我们需要理解这些区域的组织和形成过程。第一步是针对非玩具示例的网络设计并实现精确区域枚举算法。在本工作中，我们提出了深度（及浅层）神经网络中精确枚举的并行算法。我们的工作主要有三个贡献：（1）提出了一种新颖的算法框架及用于区域枚举的并行算法；（2）在多种网络架构上实现了其中一种算法，并通过实验展示了区域数量如何决定运行时间；（3）利用算法输出，展示了区域仿射变换的维度如何影响更深层对该区域的进一步划分。据我们所知，我们在比现有区域枚举文献中所有网络都更大的网络上运行了所实现的算法。此外，我们通过实验证明了并行性对任意合理规模网络区域枚举的重要性。