In this work, we investigate the unexplored intersection of domain generalization and data-free learning. In particular, we address the question: How can knowledge contained in models trained on different source data domains can be merged into a single model that generalizes well to unseen target domains, in the absence of source and target domain data? Machine learning models that can cope with domain shift are essential for for real-world scenarios with often changing data distributions. Prior domain generalization methods typically rely on using source domain data, making them unsuitable for private decentralized data. We define the novel problem of Data-Free Domain Generalization (DFDG), a practical setting where models trained on the source domains separately are available instead of the original datasets, and investigate how to effectively solve the domain generalization problem in that case. We propose DEKAN, an approach that extracts and fuses domain-specific knowledge from the available teacher models into a student model robust to domain shift. Our empirical evaluation demonstrates the effectiveness of our method which achieves first state-of-the-art results in DFDG by significantly outperforming ensemble and data-free knowledge distillation baselines.

翻译：在这项工作中,我们调查了域通用和无数据学习的未探索交叉点。我们特别探讨了以下问题:在没有源和目标域数据的情况下,如何将不同源数据领域培训模型中所包含的知识整合成一个单一模型,该模型能够向无形目标域广泛推广?能够应对域变的机械学习模型对于现实世界情景和数据分配经常变化的情景至关重要。前域通用方法通常依赖于使用源域数据,使其不适合于私人分散的数据。我们界定了数据无域化这一新问题,即数据无域通用(DFDDDG)这一实际设置,即单独提供源域培训模型,而不是原始数据集,并调查如何有效解决域化问题。我们建议DEKAN,这是一种从现有教师模型中提取和整合特定领域知识的方法,可以将特定领域知识纳入一个强有力的域变模式。我们的经验评估表明,通过显著超过可编集和无数据知识蒸馏基准,我们实现DDG第一个状态成果的方法的有效性。