We propose a novel layer-wise parameterization for convolutional neural networks (CNNs) that includes built-in robustness guarantees by enforcing a prescribed Lipschitz bound. Each layer in our parameterization is designed to satisfy a linear matrix inequality (LMI), which in turn implies dissipativity with respect to a specific supply rate. Collectively, these layer-wise LMIs ensure Lipschitz boundedness for the input-output mapping of the neural network, yielding a more expressive parameterization than through spectral bounds or orthogonal layers. Our new method LipKernel directly parameterizes dissipative convolution kernels using a 2-D Roesser-type state space model. This means that the convolutional layers are given in standard form after training and can be evaluated without computational overhead. In numerical experiments, we show that the run-time using our method is orders of magnitude faster than state-of-the-art Lipschitz-bounded networks that parameterize convolutions in the Fourier domain, making our approach particularly attractive for improving the robustness of learning-based real-time perception or control in robotics, autonomous vehicles, or automation systems. We focus on CNNs, and in contrast to previous works, our approach accommodates a wide variety of layers typically used in CNNs, including 1-D and 2-D convolutional layers, maximum and average pooling layers, as well as strided and dilated convolutions and zero padding. However, our approach naturally extends beyond CNNs as we can incorporate any layer that is incrementally dissipative.

翻译：我们提出了一种新颖的卷积神经网络（CNN）逐层参数化方法，通过强制施加预设的Lipschitz界，内置了鲁棒性保证。参数化中的每一层均被设计为满足线性矩阵不等式（LMI），进而相对于特定供给率满足耗散性。这些逐层LMI共同确保了神经网络输入-输出映射的Lipschitz有界性，相比通过谱界或正交层实现参数化，该方法具有更强的表达能力。我们的新方法LipKernel直接采用二维Roesser型状态空间模型参数化耗散卷积核。这意味着卷积层在训练后以标准形式呈现，且评估时无额外计算开销。数值实验表明，使用我们的方法运行时比当前最先进的、在傅里叶域参数化卷积的Lipschitz有界网络快数个数量级，这使得该方法对提升机器人、自动驾驶车辆或自动化系统中基于学习的实时感知或控制的鲁棒性尤为具有吸引力。我们聚焦于CNN，与以往工作不同，我们的方法适用于CNN中常用的多种层类型，包括一维和二维卷积层、最大池化和平均池化层，以及步长卷积、空洞卷积和零填充。然而，我们的方法可自然扩展到CNN之外，因为我们可以纳入任何增量耗散的层。