Due to domain shifts, machine learning systems typically struggle to generalize well to new domains that differ from those of training data, which is what domain generalization (DG) aims to address. Although a variety of DG methods have been proposed, most of them fall short in interpretability and require domain labels, which are not available in many real-world scenarios. This paper presents a novel DG method, called HMOE: Hypernetwork-based Mixture of Experts (MoE), which does not rely on domain labels and is more interpretable. MoE proves effective in identifying heterogeneous patterns in data. For the DG problem, heterogeneity arises exactly from domain shifts. HMOE employs hypernetworks taking vectors as input to generate the weights of experts, which promotes knowledge sharing among experts and enables the exploration of their similarities in a low-dimensional vector space. We benchmark HMOE against other DG methods under a fair evaluation framework -- DomainBed. Our extensive experiments show that HMOE can effectively separate mixed-domain data into distinct clusters that are surprisingly more consistent with human intuition than original domain labels. Using self-learned domain information, HMOE achieves state-of-the-art results on most datasets and significantly surpasses other DG methods in average accuracy across all datasets.

翻译：由于领域偏移，机器学习系统通常难以泛化到与训练数据不同的新领域，而领域泛化（DG）正是为了解决这一问题。尽管已有多种DG方法被提出，但大多数方法缺乏可解释性，且需要领域标签，这在许多现实场景中难以获取。本文提出一种新颖的DG方法——HMOE：超网络驱动的专家混合模型（MoE），该方法无需依赖领域标签，且具有更高的可解释性。MoE在识别数据中的异质模式方面表现优异。对于DG问题，异质性恰恰源于领域偏移。HMOE采用超网络以向量为输入来生成专家权重，这促进了专家间的知识共享，并能在低维向量空间中探索专家间的相似性。我们在公平评估框架DomainBed下将HMOE与其他DG方法进行基准测试。大量实验表明，HMOE能有效将混合领域数据分离为不同的聚类簇，这些聚类簇与原始领域标签相比，惊人地与人类直觉更为一致。通过利用自学习的领域信息，HMOE在大多数数据集上取得了当前最优结果，并在所有数据集上的平均准确率显著超越其他DG方法。