This work introduces the first toolkit around path-norms that is fully able to encompass general DAG ReLU networks with biases, skip connections and any operation based on the extraction of order statistics: max pooling, GroupSort etc. This toolkit notably allows us to establish generalization bounds for modern neural networks that are not only the most widely applicable path-norm based ones, but also recover or beat the sharpest known bounds of this type. These extended path-norms further enjoy the usual benefits of path-norms: ease of computation, invariance under the symmetries of the network, and improved sharpness on feedforward networks compared to the product of operators' norms, another complexity measure most commonly used. The versatility of the toolkit and its ease of implementation allow us to challenge the concrete promises of path-norm-based generalization bounds, by numerically evaluating the sharpest known bounds for ResNets on ImageNet.

翻译：本文提出了首个能够完全涵盖带有偏置、跳跃连接及基于顺序统计量提取操作（如最大池化、GroupSort等）的通用DAG ReLU网络的路径范数工具包。该工具包不仅使我们能够为现代神经网络建立适用范围最广的基于路径范数的泛化界，还能恢复或超越此类方法已知的最精确界。这些扩展的路径范数进一步继承了路径范数的常规优势：易于计算、对网络对称性具有不变性，以及在前馈网络上相比另一种常用复杂度度量——运算符范数乘积——具有更优的尖锐性。该工具包的通用性和易实现性使我们能够通过数值评估ResNet在ImageNet上已知最精确的泛化界，从而挑战基于路径范数的泛化界的具体前景。