We prove that the set of functions representable by ReLU neural networks with integer weights strictly increases with the network depth while allowing arbitrary width. More precisely, we show that $\lceil\log_2(n)\rceil$ hidden layers are indeed necessary to compute the maximum of $n$ numbers, matching known upper bounds. Our results are based on the known duality between neural networks and Newton polytopes via tropical geometry. The integrality assumption implies that these Newton polytopes are lattice polytopes. Then, our depth lower bounds follow from a parity argument on the normalized volume of faces of such polytopes.

翻译：我们证明，在允许任意宽度的情况下，具有整权重的ReLU神经网络可表示的函数集严格随网络深度增加而增大。具体而言，我们表明计算n个数的最大值确实需要$\lceil\log_2(n)\rceil$个隐藏层，这与已知上界相匹配。我们的结果基于热带几何中神经网络与牛顿多胞体之间的已知对偶关系。整值假设意味着这些牛顿多胞体是格多胞体。随后，我们的深度下界通过此类多胞体规范体积的奇偶性论证推导得出。