There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated classifier vectors of all flat classes collapse to the vertices of a simplex Equiangular Tight Frame (ETF). Recent work has tried to exploit this phenomenon by fixing the related classifier weights to a pre-computed ETF to induce neural collapse and maximize the separation of the learned features when training with imbalanced data. In this work, we propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame), instead of an ETF, and use a cosine similarity-based auxiliary loss to learn hierarchy-aware penultimate features that collapse to the HAFrame. We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets of varying scales with hierarchies of heights ranging from 3 to 12. Code: https://github.com/ltong1130ztr/HAFrame

翻译：最近发现了一种名为“神经坍缩”的引人现象：在深度神经网络分类任务的训练末期，所有平级类别的类内倒数第二层特征均值与对应分类器向量会坍缩至单纯形等角紧框架的顶点。近期研究尝试通过将相关分类器权重固定为预计算的等角紧框架来诱导神经坍缩，从而在非平衡数据训练时最大化学习特征的分离性。本文提出将深度神经网络的线性分类器固定为层次感知框架（HAFrame）而非等角紧框架，并引入基于余弦相似度的辅助损失函数，使学习到的层次感知倒数第二层特征坍缩至该层次感知框架。实验表明，在多个规模不同、层次高度为3至12的数据集上，本方法在保持模型top-1准确率的同时有效降低了预测错误严重性。代码：https://github.com/ltong1130ztr/HAFrame