Generalized category discovery (GCD) aims at grouping unlabeled samples from known and unknown classes, given labeled data of known classes. To meet the recent decentralization trend in the community, we introduce a practical yet challenging task, namely Federated GCD (Fed-GCD), where the training data are distributively stored in local clients and cannot be shared among clients. The goal of Fed-GCD is to train a generic GCD model by client collaboration under the privacy-protected constraint. The Fed-GCD leads to two challenges: 1) representation degradation caused by training each client model with fewer data than centralized GCD learning, and 2) highly heterogeneous label spaces across different clients. To this end, we propose a novel Associated Gaussian Contrastive Learning (AGCL) framework based on learnable GMMs, which consists of a Client Semantics Association (CSA) and a global-local GMM Contrastive Learning (GCL). On the server, CSA aggregates the heterogeneous categories of local-client GMMs to generate a global GMM containing more comprehensive category knowledge. On each client, GCL builds class-level contrastive learning with both local and global GMMs. The local GCL learns robust representation with limited local data. The global GCL encourages the model to produce more discriminative representation with the comprehensive category relationships that may not exist in local data. We build a benchmark based on six visual datasets to facilitate the study of Fed-GCD. Extensive experiments show that our AGCL outperforms the FedAvg-based baseline on all datasets.

翻译：广义类别发现（GCD）旨在利用已知类别的标注数据，将来自已知和未知类别的未标注样本进行聚类。为适应该领域近期去中心化的发展趋势，我们提出了一项兼具实用性与挑战性的新任务——联邦广义类别发现（Fed-GCD），其中训练数据分布式存储在本地客户端，且无法在各客户端之间共享。Fed-GCD的目标是在隐私保护约束下，通过客户端协作训练通用的GCD模型。该任务面临两大挑战：1）相较于集中式GCD学习，各客户端模型因训练数据量减少而导致的表征退化；2）不同客户端间高度异构的标签空间。为此，我们提出了一种基于可学习高斯混合模型（GMM）的新型关联高斯对比学习（AGCL）框架，包含客户端语义关联（CSA）和全局-局部GMM对比学习（GCL）两个模块。在服务器端，CSA聚合本地客户端GMM的异构类别以生成包含更全面类别知识的全局GMM；在各客户端上，GCL同时利用局部和全局GMM构建类别级对比学习：局部GCL在有限的本地数据上学习鲁棒表征，全局GCL则通过本地数据中可能不存在的全面类别关系，引导模型生成更具判别性的表征。我们基于六个视觉数据集构建了基准测试套件以促进Fed-GCD研究。大量实验表明，我们的AGCL在所有数据集上均优于基于FedAvg的基线方法。