Real-world datasets exhibit imbalances of varying types and degrees. Several techniques based on re-weighting and margin adjustment of loss are often used to enhance the performance of neural networks, particularly on minority classes. In this work, we analyze the class-imbalanced learning problem by examining the loss landscape of neural networks trained with re-weighting and margin-based techniques. Specifically, we examine the spectral density of Hessian of class-wise loss, through which we observe that the network weights converge to a saddle point in the loss landscapes of minority classes. Following this observation, we also find that optimization methods designed to escape from saddle points can be effectively used to improve generalization on minority classes. We further theoretically and empirically demonstrate that Sharpness-Aware Minimization (SAM), a recent technique that encourages convergence to a flat minima, can be effectively used to escape saddle points for minority classes. Using SAM results in a 6.2\% increase in accuracy on the minority classes over the state-of-the-art Vector Scaling Loss, leading to an overall average increase of 4\% across imbalanced datasets. The code is available at: https://github.com/val-iisc/Saddle-LongTail.

翻译：在这项工作中,我们通过审查经过再加权和边际技术培训的神经网络的流失情况来分析班级平衡学习问题。具体地说,我们研究了Hessian的舱级损失的光谱密度,我们通过这种研究发现,网络重量会汇合到少数阶层损失地貌的一个峰值点。根据这项观察,我们还发现,可以有效地使用旨在逃离马鞍点的优化方法来改进少数民族班级的常识化。我们进一步从理论上和从经验上证明,鼓励与平板小型学校融合的一种最新技术,即鼓励与平板小型学校趋同的一种最新技术,可以有效地摆脱少数群体班级的悬崖点。我们利用SAM的结果发现,在最先进的矢量缩损失中,少数民族班级的精度增加了6.2 ⁇ 。我们发现,通过这一观察,可以使整个不平衡的数据集平均增加4 ⁇ 。这个代码可以查到: https://giuthus/ailbth。