There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated classifier vectors of all flat classes collapse to the vertices of a simplex Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon by fixing the related classifier weights to a pre-computed ETF to induce neural collapse and maximize the separation of the learned features when training with imbalanced data. In this work, we propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF, and use a cosine similarity-based auxiliary loss to learn hierarchy-aware penultimate features that collapse to the HAFrame. We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets of varying scales with hierarchies of heights ranging from 3 to 12. We will release our code on GitHub in the near future.

翻译：近期发现一个名为神经坍塌的有趣现象：在深度神经网络分类训练的终端阶段，各类别的倒数第二层特征均值及其对应的分类器向量会坍塌至单纯形等角紧框架（ETF）的顶点。现有研究尝试通过将相关分类器权重固定为预计算ETF来诱导神经坍塌，从而在不平衡数据训练时最大化所学特征的分离度。本文提出将深度神经网络的线性分类器固定为层级感知框架（HAFrame）而非ETF，并采用基于余弦相似度的辅助损失函数，使倒数第二层特征学习层级感知属性并坍塌至HAFrame。实验证明，在多个包含3至12层层级结构的不同规模数据集上，该方法在保持top-1准确率的同时，有效降低了模型预测的错误严重性。代码将于近期在GitHub上开源。