Data imbalance and open-ended distribution are two intrinsic characteristics of the real visual world. Though encouraging progress has been made in tackling each challenge separately, few works dedicated to combining them towards real-world scenarios. While several previous works have focused on classifying close-set samples and detecting open-set samples during testing, it's still essential to be able to classify unknown subjects as human beings. In this paper, we formally define a more realistic task as distribution-agnostic generalized category discovery (DA-GCD): generating fine-grained predictions for both close- and open-set classes in a long-tailed open-world setting. To tackle the challenging problem, we propose a Self-Balanced Co-Advice contrastive framework (BaCon), which consists of a contrastive-learning branch and a pseudo-labeling branch, working collaboratively to provide interactive supervision to resolve the DA-GCD task. In particular, the contrastive-learning branch provides reliable distribution estimation to regularize the predictions of the pseudo-labeling branch, which in turn guides contrastive learning through self-balanced knowledge transfer and a proposed novel contrastive loss. We compare BaCon with state-of-the-art methods from two closely related fields: imbalanced semi-supervised learning and generalized category discovery. The effectiveness of BaCon is demonstrated with superior performance over all baselines and comprehensive analysis across various datasets. Our code is publicly available.
翻译:数据不平衡与开放分布是真实视觉世界的两个固有特性。尽管分别应对每个挑战已取得鼓舞人心的进展,但鲜有研究致力于将二者结合以适配真实场景。虽然先前若干工作聚焦于测试阶段对封闭集样本进行分类并检测开放集样本,但能够像人类一样对未知主体进行分类仍至关重要。本文正式定义了一个更贴近现实的任务——分布无关的广义类别发现(DA-GCD):在长尾开放世界场景中为封闭集和开放集类别生成细粒度预测。为解决这一挑战性问题,我们提出自平衡协同指导对比框架(BaCon),该框架由对比学习分支和伪标签分支协同工作,通过交互式监督机制解决DA-GCD任务。具体而言,对比学习分支提供可靠的分布估计以正则化伪标签分支的预测,而伪标签分支则通过自平衡知识迁移及所提出的新型对比损失函数反向引导对比学习。我们将BaCon与来自两个密切相关领域(不平衡半监督学习和广义类别发现)的最先进方法进行对比。通过在所有基线上展现的卓越性能及跨多个数据集的综合分析,BaCon的有效性得到充分证明。我们的代码已开源。