Due to domain shift, machine learning systems typically fail to generalize well to domains different from those of training data, which is what domain generalization (DG) aims to address. Although various DG methods have been developed, most of them lack interpretability and require domain labels that are not available in many real-world scenarios. This paper presents a novel DG method, called HMOE: Hypernetwork-based Mixture of Experts (MoE), which does not rely on domain labels and is more interpretable. MoE proves effective in identifying heterogeneous patterns in data. For the DG problem, heterogeneity arises exactly from domain shift. HMOE uses hypernetworks taking vectors as input to generate experts' weights, which allows experts to share useful meta-knowledge and enables exploring experts' similarities in a low-dimensional vector space. We compare HMOE with other DG algorithms under a fair and unified benchmark-DomainBed. Our extensive experiments show that HMOE can divide mixed-domain data into distinct clusters that are surprisingly more consistent with human intuition than original domain labels. Compared to other DG methods, HMOE shows competitive performance and achieves SOTA results in some cases.

翻译：由于领域偏移，机器学习系统通常难以泛化到与训练数据不同的领域，这正是领域泛化（DG）旨在解决的问题。尽管已开发出多种DG方法，但大多数方法缺乏可解释性，且依赖于在许多现实场景中无法获得的领域标签。本文提出了一种新颖的DG方法——HMOE：基于超网络的混合专家模型（MoE），该方法不依赖领域标签且更具可解释性。MoE在识别数据中的异质模式方面表现出有效性。对于DG问题，异质性恰由领域偏移产生。HMOE使用以向量为输入的超网络生成专家权重，使专家能够共享有用的元知识，并能在低维向量空间中探索专家间的相似性。我们在公平统一的基准平台DomainBed上，将HMOE与其他DG算法进行了比较。大量实验表明，HMOE能够将混合领域的数据划分为不同的聚类，这些聚类与原始领域标签相比，出人意料地与人类直觉更为一致。与其他DG方法相比，HMOE展现出具有竞争力的性能，并在某些情况下达到了最优结果。