We show the effectiveness of automatic differentiation in efficiently and correctly computing and controlling the spectrum of implicitly linear operators, a rich family of layer types including all standard convolutional and dense layers. We provide the first clipping method which is correct for general convolution layers, and illuminate the representational limitation that caused correctness issues in prior work. We study the effect of the batch normalization layers when concatenated with convolutional layers and show how our clipping method can be applied to their composition. By comparing the accuracy and performance of our algorithms to the state-of-the-art methods, using various experiments, we show they are more precise and efficient and lead to better generalization and adversarial robustness. We provide the code for using our methods at https://github.com/Ali-E/FastClip.

翻译：我们证明了自动微分在高效且正确地计算和控制隐式线性算子谱方面的有效性，这类算子涵盖包括所有标准卷积层和稠密层在内的丰富层类型。我们提出了首个适用于一般卷积层的正确裁剪方法，并阐明了导致先前工作存在正确性问题的表示局限性。我们研究了批归一化层与卷积层级联时的效应，并展示了如何将我们的裁剪方法应用于其组合结构。通过在不同实验中对比我们的算法与现有最优方法的精度和性能，我们证明其具有更高的精确度和效率，并能带来更好的泛化能力和对抗鲁棒性。相关方法的代码发布于 https://github.com/Ali-E/FastClip。