Data imbalance and open-ended distribution are two intrinsic characteristics of the real visual world. Though encouraging progress has been made in tackling each challenge separately, few works dedicated to combining them towards real-world scenarios. While several previous works have focused on classifying close-set samples and detecting open-set samples during testing, it's still essential to be able to classify unknown subjects as human beings. In this paper, we formally define a more realistic task as distribution-agnostic generalized category discovery (DA-GCD): generating fine-grained predictions for both close- and open-set classes in a long-tailed open-world setting. To tackle the challenging problem, we propose a Self-Balanced Co-Advice contrastive framework (BaCon), which consists of a contrastive-learning branch and a pseudo-labeling branch, working collaboratively to provide interactive supervision to resolve the DA-GCD task. In particular, the contrastive-learning branch provides reliable distribution estimation to regularize the predictions of the pseudo-labeling branch, which in turn guides contrastive learning through self-balanced knowledge transfer and a proposed novel contrastive loss. We compare BaCon with state-of-the-art methods from two closely related fields: imbalanced semi-supervised learning and generalized category discovery. The effectiveness of BaCon is demonstrated with superior performance over all baselines and comprehensive analysis across various datasets. Our code is publicly available.
翻译:数据不平衡与开放分布是真实视觉世界的两个内在特征。尽管分别应对每个挑战已取得令人鼓舞的进展,但很少有工作致力于将两者结合以应对真实场景。虽然先前的一些研究专注于在测试时分类闭集样本并检测开集样本,但像人类一样对未知对象进行分类仍至关重要。本文正式定义了一个更具现实性的任务——分布无关的广义类别发现(DA-GCD):在长尾开放世界场景中,对闭集和开集类别生成细粒度预测。为应对这一挑战性难题,我们提出了一种自平衡协同对比框架(BaCon),该框架由对比学习分支和伪标签分支组成,通过协作提供交互式监督来解决DA-GCD任务。具体而言,对比学习分支提供可靠的分布估计以正则化伪标签分支的预测,而伪标签分支则通过自平衡知识转移及提出的新型对比损失反向引导对比学习。我们将BaCon与两个密切相关领域(不平衡半监督学习与广义类别发现)的最新方法进行了比较。通过在多种数据集上的全面分析和优于所有基线的卓越性能,证明了BaCon的有效性。我们的代码已公开。