This work introduces the first toolkit around path-norms that is fully able to encompass general DAG ReLU networks with biases, skip connections and max pooling. This toolkit notably allows us to establish generalization bounds for real modern neural networks that are not only the most widely applicable path-norm based ones, but also recover or beat the sharpest known bounds of this type. These extended path-norms further enjoy the usual benefits of path-norms: ease of computation, invariance under the symmetries of the network, and improved sharpness on feedforward networks compared to the product of operators' norms, another complexity measure most commonly used. The versatility of the toolkit and its ease of implementation allow us to challenge the concrete promises of path-norm-based generalization bounds, by numerically evaluating the sharpest known bounds for ResNets on ImageNet.

翻译：本文首次提出了一套完整的路径范数工具包，能够涵盖带有偏置、跳跃连接和最大池化的一般DAG ReLU网络。该工具包不仅使我们能够为真实现代神经网络建立基于路径范数的泛化界（这些界是此类方法中适用范围最广的），还能恢复或超越该类方法现有最陡的已知界。这些扩展的路径范数继承了路径范数的传统优势：易于计算、对网络对称性具有不变性，且与前馈网络中常用的另一种复杂度度量——算子范数乘积相比，具有更优的陡峭性。该工具包的通用性与易实现性，使我们能够通过数值评估ImageNet上ResNet最陡已知界，来检验基于路径范数的泛化界的实际应用前景。