We consider the problem of OOD generalization, where the goal is to train a model that performs well on test distributions that are different from the training distribution. Deep learning models are known to be fragile to such shifts and can suffer large accuracy drops even for slightly different test distributions. We propose a new method - DAFT - based on the intuition that adversarially robust combination of a large number of rich features should provide OOD robustness. Our method carefully distills the knowledge from a powerful teacher that learns several discriminative features using standard training while combining them using adversarial training. The standard adversarial training procedure is modified to produce teachers which can guide the student better. We evaluate DAFT on standard benchmarks in the DomainBed framework, and demonstrate that DAFT achieves significant improvements over the current state-of-the-art OOD generalization methods. DAFT consistently out-performs well-tuned ERM and distillation baselines by up to 6%, with more pronounced gains for smaller networks.

翻译：我们考虑OOD的概括化问题,我们的目标是在与培训分布不同的测试分布上培养一个运行良好的模型。深层次的学习模式已知在这种转变中脆弱不堪,即使测试分布略有不同,也可能受到大量精度下降的影响。我们提出一种新的方法DAFT,其依据的直觉是,大量丰富特征的对抗性强力结合应该提供OOD的稳健性。我们的方法仔细地从一个强势教师那里提取知识,该教师使用标准培训来学习几种歧视性特征,同时使用对抗性培训将其结合起来。标准对抗性培训程序经过修改,培养能够更好地指导学生的教师。我们根据DOAFT框架的标准基准对DAFT进行了评估,并证明DAFT在目前最先进的OOD通用方法上取得了重大改进。DAFT一贯将调整良好的机构风险管理和蒸馏基线提高6%,对较小的网络则有更显著的收益。