Invariances in neural networks are useful and necessary for many tasks. However, the representation of the invariance of most neural network models has not been characterized. We propose measures to quantify the invariance of neural networks in terms of their internal representation. The measures are efficient and interpretable, and can be applied to any neural network model. They are also more sensitive to invariance than previously defined measures. We validate the measures and their properties in the domain of affine transformations and the CIFAR10 and MNIST datasets, including their stability and interpretability. Using the measures, we perform a first analysis of CNN models and show that their internal invariance is remarkably stable to random weight initializations, but not to changes in dataset or transformation. We believe the measures will enable new avenues of research in invariance representation.

翻译：神经网络中的不变性对许多任务而言既实用又必要。然而，大多数神经网络模型的不变性表征尚未明确。我们提出了一系列度量方法，用于量化神经网络在其内部表示中的不变性。这些度量兼具高效性与可解释性，可应用于任意神经网络模型，并且相较于先前定义的不变性度量具有更高的敏感性。我们通过仿射变换及CIFAR10和MNIST数据集验证了这些度量及其特性，包括其稳定性和可解释性。利用这些度量，我们对CNN模型进行了初步分析，结果表明其内部不变性对随机权重初始化具有显著的稳定性，但对数据集或变换的变动较为敏感。我们相信这些度量将为不变性表征研究开辟新的方向。