This work introduces the first toolkit around path-norms that fully encompasses general DAG ReLU networks with biases, skip connections and any operation based on the extraction of order statistics: max pooling, GroupSort etc. This toolkit notably allows us to establish generalization bounds for modern neural networks that are not only the most widely applicable path-norm based ones, but also recover or beat the sharpest known bounds of this type. These extended path-norms further enjoy the usual benefits of path-norms: ease of computation, invariance under the symmetries of the network, and improved sharpness on layered fully-connected networks compared to the product of operator norms, another complexity measure most commonly used. The versatility of the toolkit and its ease of implementation allow us to challenge the concrete promises of path-norm-based generalization bounds, by numerically evaluating the sharpest known bounds for ResNets on ImageNet.

翻译：本文首次引入了一个全面涵盖带有偏置、跳跃连接及基于顺序统计量提取操作（如最大池化、GroupSort等）的通用有向无环图ReLU网络的路径范数工具包。该工具包不仅能够为现代神经网络建立适用范围最广的基于路径范数的泛化界，还能恢复甚至超越已知同类结论的最紧结果。这些扩展路径范数进一步继承了路径范数的传统优势：易于计算、在网络对称变换下保持不变性，以及相比另一种常用复杂度度量——算子范数乘积——在全连接分层网络上具有更优的紧致性。该工具包的通用性与简便实现，使我们能够通过数值评估ImageNet上ResNet已知最紧泛化界，进而质疑基于路径范数的泛化界在实际应用中的具体承诺。