Neural network representations contain structure beyond what was present in the training labels. For instance, representations of images that are visually or semantically similar tend to lie closer to each other than to dissimilar images, regardless of their labels. Clustering these representations can thus provide insights into dataset properties as well as the network internals. In this work, we study how the many design choices involved in neural network training affect the clusters formed in the hidden representations. To do so, we establish an evaluation setup based on the BREEDS hierarchy, for the task of subclass clustering after training models with only superclass information. We isolate the training dataset and architecture as important factors affecting clusterability. Datasets with labeled classes consisting of unrelated subclasses yield much better clusterability than those following a natural hierarchy. When using pretrained models to cluster representations on downstream datasets, models pretrained on subclass labels provide better clusterability than models pretrained on superclass labels, but only when there is a high degree of domain overlap between the pretraining and downstream data. Architecturally, we find that normalization strategies affect which layers yield the best clustering performance, and, surprisingly, Vision Transformers attain lower subclass clusterability than ResNets.

翻译：神经网络表示包含超出训练标签所呈现的结构。例如，视觉或语义相似的图像，无论其标签如何，其表示往往比不相似图像的表示更接近彼此。因此，对这些表示进行聚类可以揭示数据集属性及网络内部结构。在本文中，我们研究了神经网络训练中涉及的众多设计选择如何影响隐藏表示中形成的聚类。为此，我们基于BREEDS层级建立了一个评估框架，用于在仅使用超类信息训练模型后对子类进行聚类。我们将训练数据集和架构视为影响聚类能力的关键因素。由不相关子类组成的带标签类的数据集，其聚类能力远优于遵循自然层级结构的数据集。当使用预训练模型对下游数据集的表示进行聚类时，在子类标签上预训练的模型比在超类标签上预训练的模型能提供更好的聚类能力，但这仅当预训练数据与下游数据之间存在高度领域重叠时成立。在架构方面，我们发现归一化策略会影响哪些层能达到最佳聚类性能，并且令人惊讶的是，Vision Transformers的子类聚类能力低于ResNets。