This paper introduces a new parameterization of deep neural networks (both fully-connected and convolutional) with guaranteed Lipschitz bounds, i.e. limited sensitivity to perturbations. The Lipschitz guarantees are equivalent to the tightest-known bounds based on certification via a semidefinite program (SDP), which does not scale to large models. In contrast to the SDP approach, we provide a ``direct'' parameterization, i.e. a smooth mapping from $\mathbb R^N$ onto the set of weights of Lipschitz-bounded networks. This enables training via standard gradient methods, without any computationally intensive projections or barrier terms. The new parameterization can equivalently be thought of as either a new layer type (the \textit{sandwich layer}), or a novel parameterization of standard feedforward networks with parameter sharing between neighbouring layers. We illustrate the method with some applications in image classification (MNIST and CIFAR-10).

翻译：本文提出了一种新的深度神经网络（包括全连接网络和卷积网络）参数化方法，该方法保证了Lipschitz界（即对扰动的有限敏感性）。这些Lipschitz保证等价于基于半定规划（SDP）认证的已知最紧界，但SDP方法无法扩展至大规模模型。与SDP方法不同，我们提供了一种“直接”参数化，即从$\mathbb R^N$到Lipschitz有界网络权重集合的光滑映射。这使得可以通过标准梯度方法进行训练，无需高计算代价的投影或障碍项。这种新的参数化可等价地视为一种新的层类型（\textit{三明治层}），或是一种相邻层间参数共享的标准前馈网络的新型参数化。我们通过图像分类（MNIST和CIFAR-10）中的应用验证了该方法。