Domain generalization (DG) aims to learn a generalizable model from multiple training domains such that it can perform well on unseen target domains. A popular strategy is to augment training data to benefit generalization through methods such as Mixup~\cite{zhang2018mixup}. While the vanilla Mixup can be directly applied, theoretical and empirical investigations uncover several shortcomings that limit its performance. Firstly, Mixup cannot effectively identify the domain and class information that can be used for learning invariant representations. Secondly, Mixup may introduce synthetic noisy data points via random interpolation, which lowers its discrimination capability. Based on the analysis, we propose a simple yet effective enhancement for Mixup-based DG, namely domain-invariant Feature mIXup (FIX). It learns domain-invariant representations for Mixup. To further enhance discrimination, we leverage existing techniques to enlarge margins among classes to further propose the domain-invariant Feature MIXup with Enhanced Discrimination (FIXED) approach. We present theoretical insights about guarantees on its effectiveness. Extensive experiments on seven public datasets across two modalities including image classification (Digits-DG, PACS, Office-Home) and time series (DSADS, PAMAP2, UCI-HAR, and USC-HAD) demonstrate that our approach significantly outperforms nine state-of-the-art related methods, beating the best performing baseline by 6.5\% on average in terms of test accuracy. Code is available at: https://github.com/jindongwang/transferlearning/tree/master/code/deep/fixed.
翻译:域泛化(DG)旨在从多个训练域中学习一个可泛化的模型,使其能在未见过的目标域上表现良好。一种流行的策略是通过混合等方法扩充训练数据以促进泛化。尽管原始混合可直接应用,但理论和实验研究揭示了若干限制其性能的缺陷。首先,混合无法有效识别可用于学习不变表征的域和类别信息。其次,混合可能通过随机插值引入合成的噪声数据点,从而降低其判别能力。基于此分析,我们提出了一种简单而有效的基于混合的域泛化增强方法,即域不变特征混合。它学习用于混合的域不变表征。为进一步增强判别能力,我们利用现有技术扩大类别间间隔,进而提出具有增强判别能力的域不变特征混合方法。我们提供了关于其有效性保证的理论见解。在两个模态(包括图像分类:Digits-DG、PACS、Office-Home,以及时间序列:DSADS、PAMAP2、UCI-HAR、USC-HAD)的七个公开数据集上的大量实验表明,我们的方法显著优于九种最先进的相关方法,在测试准确率上平均超过最优基线6.5%。代码见:https://github.com/jindongwang/transferlearning/tree/master/code/deep/fixed。