Understanding the relationship between the depth of a neural network and its representational capacity is a central problem in deep learning theory. In this work, we develop a geometric framework to analyze the expressivity of ReLU networks with the notion of depth complexity for convex polytopes. The depth of a polytope recursively quantifies the number of alternating convex hull and Minkowski sum operations required to construct it. This geometric perspective serves as a rigorous tool for deriving depth lower bounds and understanding the structural limits of deep neural architectures. We establish lower and upper bounds on the depth of polytopes, as well as tight bounds for classical families. These results yield two main consequences. First, we provide a purely geometric proof of the expressivity bound by Arora et al. (2018), confirming that $\lceil \log_2(n+1)\rceil$ hidden layers suffice to represent any continuous piecewise linear (CPWL) function. Second, we prove that, unlike general ReLU networks, convex polytopes do not admit a universal depth bound. Specifically, the depth of cyclic polytopes in dimensions $n \geq 4$ grows unboundedly with the number of vertices. This result implies that Input Convex Neural Networks (ICNNs) cannot represent all convex CPWL functions with a fixed depth, revealing a sharp separation in expressivity between ICNNs and standard ReLU networks.

翻译：理解神经网络深度与其表示能力之间的关系是深度学习理论中的一个核心问题。本文发展了一个几何框架，通过凸多胞体的深度复杂度概念来分析ReLU网络的表达力。多胞体的深度递归地量化了构造它所需的交替凸包和闵可夫斯基和运算的次数。这一几何视角为推导深度下界和理解深度神经架构的结构限制提供了严格工具。我们建立了多胞体深度的上下界，以及经典族类的紧界。这些结果产生两个主要推论。首先，我们给出了Arora等人(2018)表达力界的纯几何证明，确认了$\lceil \log_2(n+1)\rceil$个隐藏层足以表示任何连续分段线性(CPWL)函数。其次，我们证明了与一般ReLU网络不同，凸多胞体不具备普遍深度界。具体而言，在维数$n \geq 4$的情况下，循环多胞体的深度随顶点数量无界增长。该结果意味着输入凸神经网络(ICNNs)无法以固定深度表示所有凸CPWL函数，揭示了ICNNs与标准ReLU网络在表达力上的显著分离。