Domain generalization (DG) aims to learn a generalized model to an unseen target domain using only limited source domains. Previous attempts to DG fail to learn domain-invariant representations only from the source domains due to the significant domain shifts between training and test domains. Instead, we re-formulate the DG objective using mutual information with the oracle model, a model generalized to any possible domain. We derive a tractable variational lower bound via approximating the oracle model by a pre-trained model, called Mutual Information Regularization with Oracle (MIRO). Our extensive experiments show that MIRO significantly improves the out-of-distribution performance. Furthermore, our scaling experiments show that the larger the scale of the pre-trained model, the greater the performance improvement of MIRO. Source code is available at https://github.com/kakaobrain/miro.

翻译：广域化(DG) 旨在只使用有限的源域,从一个看不见的目标域学习一种通用模式。 DG以前试图从源域中学习的只是域差异表,因为培训和测试域之间的重大领域变化。相反,我们利用与甲骨文模型的相互信息来重新制定DG目标,这是向任何可能域推广的一种模式。我们通过一个预先培训的模型,即与甲骨文(MIRO)的相互信息规范化(MIRO),来获得一个可移动的更低的变式约束。我们的广泛实验显示,MIRO大大改进了分配外的性能。此外,我们的缩放实验显示,经过培训的模型规模越大,MIRO的性能改进越大。源码可在https://github.com/kaobrain/miro查阅 https://github.com/kaobrain/miro查阅。