Empirical studies have widely demonstrated that neural networks are highly sensitive to small, adversarial perturbations of the input. The worst-case robustness against these so-called adversarial examples can be quantified by the Lipschitz constant of the neural network. In this paper, we study upper and lower bounds for the Lipschitz constant of random ReLU neural networks. Specifically, we assume that the weights and biases follow a generalization of the He initialization, where general symmetric distributions for the biases are permitted. For shallow neural networks, we characterize the Lipschitz constant up to an absolute numerical constant. For deep networks with fixed depth and sufficiently large width, our established upper bound is larger than the lower bound by a factor that is logarithmic in the width.

翻译：实证研究广泛表明，神经网络对输入的小幅对抗扰动高度敏感。针对此类所谓对抗样本的最坏情况鲁棒性，可通过神经网络的Lipschitz常数来量化。本文研究了随机ReLU神经网络Lipschitz常数的上下界。具体而言，我们假设权重和偏置服从He初始化的推广形式，其中允许偏置采用一般对称分布。对于浅层神经网络，我们将Lipschitz常数刻画至绝对数值常数精度。对于固定深度且宽度足够大的深层网络，所建立的上界比下界大一个关于宽度的对数因子。