This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a certain layer in the DNN, our method disentangles feature components of different complexity orders from the feature. We further design a set of metrics to evaluate the reliability, the effectiveness, and the significance of over-fitting of these feature components. Furthermore, we successfully discover a close relationship between the feature complexity and the performance of DNNs. As a generic mathematical tool, the feature complexity and the proposed metrics can also be used to analyze the success of network compression and knowledge distillation.

翻译：本文旨在定义、量化并分析深度神经网络所学习的特征复杂度。我们提出了特征复杂度的一般性定义。给定深度神经网络中某一层的特征，我们的方法从该特征中解构出不同复杂度阶数的特征成分。我们进一步设计了一套指标来评估这些特征成分的可靠性、有效性以及过拟合显著性。此外，我们成功发现了特征复杂度与深度神经网络性能之间的密切联系。作为一种通用的数学工具，特征复杂度及其提出的指标还可用于分析网络压缩与知识蒸馏成功的原因。