An established measure of the expressive power of a given ReLU neural network is the number of linear regions into which it partitions the input space. There exist many different, non-equivalent definitions of what a linear region actually is. We systematically assess which papers use which definitions and discuss how they relate to each other. We then analyze the computational complexity of counting the number of such regions for the various definitions. Generally, this turns out to be an intractable problem. We prove NP- and #P-hardness results already for networks with one hidden layer and strong hardness of approximation results for two or more hidden layers. Finally, on the algorithmic side, we demonstrate that counting linear regions can at least be achieved in polynomial space for some common definitions.

翻译：衡量给定ReLU神经网络表达能力的一个既定标准是其将输入空间划分的线性区域数量。对于线性区域的实际定义存在许多不同且不等价的表述。我们系统评估了不同文献采用的定义方式，并讨论了这些定义之间的关联。随后针对各类定义分析了区域计数的计算复杂性。总体而言，该问题被证明是难解的：我们证明了即使对于单隐藏层网络也存在NP难与#P难的结果，而对于两个及以上隐藏层网络则存在更强的近似计算难度结果。最后在算法层面，我们证明了针对某些常见定义，线性区域计数至少可以在多项式空间内完成。