We consider a collaborative learning setting where the goal of each agent is to improve their own model by leveraging the expertise of collaborators, in addition to their own training data. To facilitate the exchange of expertise among agents, we propose a distillation-based method leveraging shared unlabeled auxiliary data, which is pseudo-labeled by the collective. Central to our method is a trust weighting scheme that serves to adaptively weigh the influence of each collaborator on the pseudo-labels until a consensus on how to label the auxiliary data is reached. We demonstrate empirically that our collaboration scheme is able to significantly boost individual models' performance in the target domain from which the auxiliary data is sampled. At the same time, it can provably mitigate the negative impact of bad models on the collective. By design, our method adeptly accommodates heterogeneity in model architectures and substantially reduces communication overhead compared to typical collaborative learning methods.

翻译：我们考虑一种协作学习场景，其中每个智能体的目标是通过利用协作者的专长以及自身训练数据来改进自身模型。为促进智能体间的专长交换，我们提出一种基于蒸馏的方法，利用共享的无标签辅助数据（由全体成员伪标记）实现协作。该方法的核心是一种信任加权机制，该机制能自适应地调节各协作者对伪标签的影响权重，直至就辅助数据的标记方式达成共识。实验表明，我们的协作方案能显著提升个体模型在辅助数据采样目标域中的性能，同时可证明性地缓解劣质模型对集体造成的负面影响。从设计上看，本方法能灵活适应模型架构的异构性，相比典型协作学习方法大幅降低通信开销。