Zero-shot learning (ZSL) aims to identify unseen classes with zero samples during training. Broadly speaking, present ZSL methods usually adopt class-level semantic labels and compare them with instance-level semantic predictions to infer unseen classes. However, we find that such existing models mostly produce imbalanced semantic predictions, i.e. these models could perform precisely for some semantics, but may not for others. To address the drawback, we aim to introduce an imbalanced learning framework into ZSL. However, we find that imbalanced ZSL has two unique challenges: (1) Its imbalanced predictions are highly correlated with the value of semantic labels rather than the number of samples as typically considered in the traditional imbalanced learning; (2) Different semantics follow quite different error distributions between classes. To mitigate these issues, we first formalize ZSL as an imbalanced regression problem which offers empirical evidences to interpret how semantic labels lead to imbalanced semantic predictions. We then propose a re-weighted loss termed Re-balanced Mean-Squared Error (ReMSE), which tracks the mean and variance of error distributions, thus ensuring rebalanced learning across classes. As a major contribution, we conduct a series of analyses showing that ReMSE is theoretically well established. Extensive experiments demonstrate that the proposed method effectively alleviates the imbalance in semantic prediction and outperforms many state-of-the-art ZSL methods. Our code is available at https://github.com/FouriYe/ReZSL-TIP23.

翻译：零样本学习旨在训练过程中无样本的情况下识别未见类别。广义上，现有零样本学习方法通常采用类别级语义标签，并将其与实例级语义预测进行对比以推断未见类别。然而，我们发现这些现有模型大多生成不平衡的语义预测，即这些模型对某些语义能精确预测，但对其他语义则可能失效。为解决这一缺陷，我们旨在将不平衡学习框架引入零样本学习。但研究发现不平衡零样本学习面临两个独特挑战：（1）其预测不平衡性与语义标签的值高度相关，而非传统不平衡学习中通常考虑的样本数量；（2）不同语义在不同类别间呈现截然不同的误差分布。为缓解这些问题，我们首先将零样本学习形式化为一个不平衡回归问题，提供经验证据解释语义标签如何导致不平衡语义预测。随后提出一种重加权损失函数——再平衡均方误差（ReMSE），该函数追踪误差分布的均值与方差，从而确保跨类别的再平衡学习。作为主要贡献，我们通过一系列分析表明ReMSE在理论上具有完善的理论基础。大量实验证明，所提方法有效缓解了语义预测中的不平衡性，并优于多种最先进的零样本学习方法。我们的代码开源在https://github.com/FouriYe/ReZSL-TIP23。