Model Weight Averaging (MWA) is a technique that seeks to enhance model's performance by averaging the weights of multiple trained models. This paper first empirically finds that 1) the vanilla MWA can benefit the class-imbalanced learning, and 2) performing model averaging in the early epochs of training yields a greater performance improvement than doing that in later epochs. Inspired by these two observations, in this paper we propose a novel MWA technique for class-imbalanced learning tasks named Iterative Model Weight Averaging (IMWA). Specifically, IMWA divides the entire training stage into multiple episodes. Within each episode, multiple models are concurrently trained from the same initialized model weight, and subsequently averaged into a singular model. Then, the weight of this average model serves as a fresh initialization for the ensuing episode, thus establishing an iterative learning paradigm. Compared to vanilla MWA, IMWA achieves higher performance improvements with the same computational cost. Moreover, IMWA can further enhance the performance of those methods employing EMA strategy, demonstrating that IMWA and EMA can complement each other. Extensive experiments on various class-imbalanced learning tasks, i.e., class-imbalanced image classification, semi-supervised class-imbalanced image classification and semi-supervised object detection tasks showcase the effectiveness of our IMWA.

翻译：模型权重平均（MWA）是一种通过对多个已训练模型的权重进行平均以提升模型性能的技术。本文首先通过实验发现：1）基础MWA方法可促进类别不平衡学习；2）在训练早期阶段进行模型平均比在后期阶段能带来更显著的性能提升。受这两项观察启发，本文针对类别不平衡学习任务提出了一种新型MWA方法——迭代模型权重平均（IMWA）。具体而言，IMWA将整个训练阶段划分为多个片段，每个片段内从相同的初始化模型权重出发并行训练多个模型，随后将其平均为单一模型。该平均模型的权重将作为下一片段的新初始化参数，由此构建迭代式学习范式。与基础MWA相比，IMWA在相同计算成本下实现了更优的性能提升。此外，IMWA可进一步增强采用指数移动平均（EMA）策略方法的性能，证明IMWA与EMA具有互补性。在类别不平衡图像分类、半监督类别不平衡图像分类及半监督目标检测等多种类别不平衡学习任务上的大量实验验证了IMWA的有效性。