A first step is taken towards understanding often observed non-robustness phenomena of deep neural net (DNN) classifiers. This is done from the perspective of Boolean functions by asking if certain sequences of Boolean functions represented by common DNN models are noise sensitive or noise stable, concepts defined in the Boolean function literature. Due to the natural randomness in DNN models, these concepts are extended to annealed and quenched versions. Here we sort out the relation between these definitions and investigate the properties of two standard DNN architectures, the fully connected and convolutional models, when initiated with Gaussian weights.
翻译:本文初步探索了深度神经网络分类器普遍存在的非鲁棒性现象。从布尔函数视角出发,通过考察常见深度神经网络模型所表示的布尔函数序列是否具有噪声敏感或噪声稳定特性(布尔函数理论中的定义),对这一现象展开研究。鉴于深度神经网络模型固有的随机性,我们将这些概念扩展至退火和淬火两种形式。本文梳理了这些定义之间的关系,并研究了两种标准深度神经网络架构——全连接模型与卷积模型——在采用高斯权重初始化时的特性。