Empirical studies have widely demonstrated that neural networks are highly sensitive to small, adversarial perturbations of the input. The worst-case robustness against these so-called adversarial examples can be quantified by the Lipschitz constant of the neural network. However, only few theoretical results regarding this quantity exist in the literature. In this paper, we initiate the study of the Lipschitz constant of random ReLU neural networks, i.e., neural networks whose weights are chosen at random and which employ the ReLU activation function. For shallow neural networks, we characterize the Lipschitz constant up to an absolute numerical constant. Moreover, we extend our analysis to deep neural networks of sufficiently large width where we prove upper and lower bounds for the Lipschitz constant. These bounds match up to a logarithmic factor that depends on the depth.

翻译：实证研究广泛表明，神经网络对输入中的微小对抗扰动高度敏感。针对这些所谓对抗样本的最坏情况鲁棒性可通过神经网络的Lipschitz常数来量化。然而，现有文献中仅有少数关于该量的理论结果。本文中，我们首次研究随机ReLU神经网络（即权重随机选取且采用ReLU激活函数的神经网络）的Lipschitz常数。对于浅层神经网络，我们给出了Lipschitz常数的一个绝对数值常数范围内的精确刻画。此外，我们将分析扩展至宽度足够大的深度神经网络，并证明了其Lipschitz常数的上界和下界。这些边界在依赖深度的对数因子内匹配。