A characterization of the representability of neural networks is relevant to comprehend their success in artificial intelligence. This study investigate two topics on ReLU neural network expressivity and their connection with a conjecture related to the minimum depth required for representing any continuous piecewise linear (CPWL) function. The topics are the minimal depth representation of the sum and max operations, as well as the exploration of polytope neural networks. For the sum operation, we establish a sufficient condition on the minimal depth of the operands to find the minimal depth of the operation. In contrast, regarding the max operation, a comprehensive set of examples is presented, demonstrating that no sufficient conditions, depending solely on the depth of the operands, would imply a minimal depth for the operation. The study also examine the minimal depth relationship between convex CPWL functions. On polytope neural networks, we investigate basic depth properties from Minkowski sums, convex hulls, number of vertices, faces, affine transformations, and indecomposable polytopes. More significant findings include depth characterization of polygons; identification of polytopes with an increasing number of vertices, exhibiting small depth and others with arbitrary large depth; and most notably, the minimal depth of simplices, which is strictly related to the minimal depth conjecture in ReLU networks.

翻译：神经网络表示能力的刻画对于理解其在人工智能领域的成功至关重要。本研究探讨了ReLU神经网络表达能力中的两个主题及其与一个关于表示任意连续分段线性（CPWL）函数所需最小深度猜想的关联。这两个主题分别是求和与取最大值运算的最小深度表示，以及对多面体神经网络的探索。针对求和运算，我们建立了关于操作数最小深度的充分条件，以确定该运算的最小深度。相比之下，对于取最大值运算，我们提出了一系列全面的示例，证明仅依赖于操作数深度的充分条件无法决定该运算的最小深度。本研究还考察了凸CPWL函数之间的最小深度关系。在多面体神经网络方面，我们从闵可夫斯基和、凸包、顶点数量、面数量、仿射变换以及不可分解多面体等角度研究了其基本深度特性。更重要的发现包括：多边形的深度刻画；识别出顶点数递增但具有较小深度的多面体，以及其他具有任意大深度的多面体；最为显著的是，单纯形的最小深度与ReLU网络中最小深度猜想存在严格关联。