Recent studies suggest that deep learning models inductive bias towards favoring simpler features may be one of the sources of shortcut learning. Yet, there has been limited focus on understanding the complexity of the myriad features that models learn. In this work, we introduce a new metric for quantifying feature complexity, based on $\mathscr{V}$-information and capturing whether a feature requires complex computational transformations to be extracted. Using this $\mathscr{V}$-information metric, we analyze the complexities of 10,000 features, represented as directions in the penultimate layer, that were extracted from a standard ImageNet-trained vision model. Our study addresses four key questions: First, we ask what features look like as a function of complexity and find a spectrum of simple to complex features present within the model. Second, we ask when features are learned during training. We find that simpler features dominate early in training, and more complex features emerge gradually. Third, we investigate where within the network simple and complex features flow, and find that simpler features tend to bypass the visual hierarchy via residual connections. Fourth, we explore the connection between features complexity and their importance in driving the networks decision. We find that complex features tend to be less important. Surprisingly, important features become accessible at earlier layers during training, like a sedimentation process, allowing the model to build upon these foundational elements.

翻译：近期研究表明，深度学习模型对简单特征的归纳偏好可能是捷径学习的根源之一。然而，对于模型学习到的众多特征复杂性的理解仍然有限。本研究基于$\mathscr{V}$-信息提出了一种量化特征复杂度的新指标，该指标能够捕捉特征提取是否需要复杂的计算变换。利用这一$\mathscr{V}$-信息度量，我们分析了从标准ImageNet训练视觉模型中提取的10,000个特征（表示为倒数第二层的方向向量）的复杂性。本研究探讨了四个关键问题：首先，我们探究了不同复杂度特征的表现形式，发现模型中存在从简单到复杂的连续特征谱。其次，我们研究了特征在训练过程中的学习时序，发现简单特征在训练早期占主导地位，而复杂特征逐渐显现。第三，我们考察了简单与复杂特征在网络中的传播路径，发现简单特征倾向于通过残差连接绕过视觉层次结构。第四，我们探索了特征复杂度与其在驱动网络决策中的重要性之间的关联，发现复杂特征的重要性往往较低。值得注意的是，重要特征在训练过程中会像沉积过程一样，在更早的网络层中变得可获取，使模型能够基于这些基础元素进行构建。