Empirical studies have widely demonstrated that neural networks are highly sensitive to small, adversarial perturbations of the input. The worst-case robustness against these so-called adversarial examples can be quantified by the Lipschitz constant of the neural network. In this paper, we study upper and lower bounds for the Lipschitz constant of random ReLU neural networks. Specifically, we assume that the weights and biases follow a generalization of the He initialization, where general symmetric distributions for the biases are permitted. For shallow neural networks, we characterize the Lipschitz constant up to an absolute numerical constant. For deep networks with fixed depth and sufficiently large width, our established bounds differ by a factor that is logarithmic in the width.

翻译：实证研究广泛证明，神经网络对输入的小幅对抗性扰动高度敏感。针对此类所谓对抗样本的最坏情况鲁棒性，可通过神经网络的Lipschitz常数进行量化。本文研究随机ReLU神经网络Lipschitz常数的上下界。具体而言，我们假设权重和偏置服从He初始化的广义形式，允许偏置采用一般对称分布。对于浅层神经网络，我们给出了精确至绝对数值常数的Lipschitz常数刻画。对于固定深度且宽度充分大的深层网络，我们建立的上下界之差仅为宽度的对数因子。