Federated Learning (FL) facilitates distributed model development to aggregate multiple confidential data sources. The information transfer among clients can be compromised by distributional differences, i.e., by non-i.i.d. data. A particularly challenging scenario is the federated model adaptation to a target client without access to annotated data. We propose Federated Adversarial Cross Training (FACT), which uses the implicit domain differences between source clients to identify domain shifts in the target domain. In each round of FL, FACT cross initializes a pair of source clients to generate domain specialized representations which are then used as a direct adversary to learn a domain invariant data representation. We empirically show that FACT outperforms state-of-the-art federated, non-federated and source-free domain adaptation models on three popular multi-source-single-target benchmarks, and state-of-the-art Unsupervised Domain Adaptation (UDA) models on single-source-single-target experiments. We further study FACT's behavior with respect to communication restrictions and the number of participating clients.
翻译:摘要:联邦学习(FL)通过聚合多个机密数据源实现分布式模型开发。客户端间的信息传递可能因分布差异(即非独立同分布数据)而受到损害。一个极具挑战性的场景是在无标注数据的情况下,将联邦模型适应到目标客户端。我们提出联邦对抗交叉训练(FACT),该方法利用源客户端之间的隐含域差异来识别目标域中的域偏移。在FL的每一轮训练中,FACT对一对源客户端进行交叉初始化,生成域专化表征,随后将其作为直接的对抗器,以学习域不变的数据表征。实验表明,FACT在三个流行的多源-单目标基准测试中优于最先进的联邦、非联邦和无源域适应模型,并在单源-单目标实验中超过最先进的无监督域适应(UDA)模型。我们进一步研究了FACT在通信限制及参与客户端数量下的行为特性。