A characterization of the representability of neural networks is relevant to comprehend their success in artificial intelligence. This study investigate two topics on ReLU neural network expressivity and their connection with a conjecture related to the minimum depth required for representing any continuous piecewise linear function (CPWL). The topics are the minimal depth representation of the sum and max operations, as well as the exploration of polytope neural networks. For the sum operation, we establish a sufficient condition on the minimal depth of the operands to find the minimal depth of the operation. In contrast, regarding the max operation, a comprehensive set of examples is presented, demonstrating that no sufficient conditions, depending solely on the depth of the operands, would imply a minimal depth for the operation. The study also examine the minimal depth relationship between convex CPWL functions. On polytope neural networks, we investigate several fundamental properties, deriving results equivalent to those of ReLU networks, such as depth inclusions and depth computation from vertices. Notably, we compute the minimal depth of simplices, which is strictly related to the minimal depth conjecture in ReLU networks.

翻译：对神经网络可表示性的刻画有助于理解其在人工智能领域的成功。本研究探讨了ReLU神经网络表达性的两个主题及其与一项猜想（关于表示任意连续分段线性函数所需的最小深度）的关联。研究主题包括求和与最大值运算的最小深度表示，以及多面体神经网络的探索。针对求和运算，我们建立了操作数最小深度的充分条件，以确定该运算的最小深度。相反，对于最大值运算，我们通过一系列综合示例表明：仅依赖操作数深度的充分条件无法推导出该运算的最小深度。研究还考察了凸连续分段线性函数之间的最小深度关系。在多面体神经网络方面，我们探究了若干基本性质，推导出与ReLU网络等价的结果，例如深度包含关系及基于顶点计算深度。值得注意的是，我们计算了单纯形的最小深度，该结果与ReLU网络中的最小深度猜想密切相关。