We propose JEDI, a multi-dataset semi-supervised learning method, which efficiently combines knowledge from multiple experts, learned on different datasets, to train and improve the performance of individual, per dataset, student models. Our approach achieves this by addressing two important problems in current machine learning research: generalization across datasets and limitations of supervised training due to scarcity of labeled data. We start with an arbitrary number of experts, pretrained on their own specific dataset, which form the initial set of student models. The teachers are immediately derived by concatenating the feature representations from the penultimate layers of the students. We then train all models in a student-teacher semi-supervised learning scenario until convergence. In our efficient approach, student-teacher training is carried out jointly and end-to-end, showing that both students and teachers improve their generalization capacity during training. We validate our approach on four video action recognition datasets. By simultaneously considering all datasets within a unified semi-supervised setting, we demonstrate significant improvements over the initial experts.

翻译：我们提出JEDI，一种多数据集半监督学习方法，该方法高效整合了在不同数据集上训练的多位专家知识，用于训练并提升各数据集独立学生模型的性能。该方法的实现通过解决当前机器学习研究中的两个重要问题达成：跨数据集的泛化能力以及因标注数据稀缺导致的监督训练局限性。我们以任意数量的预训练专家模型作为初始学生模型集合（各专家在其专属数据集上预训练），通过串联学生模型倒数第二层的特征表示直接生成教师模型。随后在师生半监督学习场景中训练所有模型直至收敛。在我们提出的高效方法中，师生训练以联合端到端方式执行，实验表明学生与教师模型在训练过程中均提升了泛化能力。我们在四个视频动作识别数据集上验证了该方法，通过在半监督统一框架下同时处理所有数据集，证明了相较于初始专家模型的显著性能提升。