Domain generalization is a challenging problem in machine learning, where the goal is to train a model that can generalize well to unseen target domains without prior knowledge of these domains. Despite the recent success of deep neural networks, there remains a lack of effective methods for domain generalization using vision transformers. In this paper, we propose a novel domain generalization technique called Robust Representation Learning with Self-Distillation (RRLD) that utilizes a combination of i) intermediate-block self-distillation and ii) augmentation-guided self-distillation to improve the generalization capabilities of transformer-based models on unseen domains. This approach enables the network to learn robust and general features that are invariant to different augmentations and domain shifts while effectively mitigating overfitting to source domains. To evaluate the effectiveness of our proposed method, we perform extensive experiments on PACS [1] and OfficeHome [2] benchmark datasets, as well as a real-world wafer semiconductor defect dataset [3]. Our results demonstrate that RRLD achieves robust and accurate generalization performance. We observe an improvement in the range of 0.3% to 2.3% over the state-of-the-art on the three datasets.

翻译：域泛化是机器学习中的一个挑战性问题，其目标是在缺乏目标域先验知识的情况下训练出能良好泛化至未见目标域的模型。尽管深度神经网络近期取得了成功，但针对视觉Transformer在域泛化领域仍缺乏有效方法。本文提出了一种名为基于自蒸馏的鲁棒表征学习（RRLD）的新型域泛化技术，该技术结合了i）中间块自蒸馏与ii）增强引导自蒸馏，以提升基于Transformer模型在未见域上的泛化能力。该方法使网络能够学习对多种数据增强和域偏移具有不变性的鲁棒通用特征，同时有效缓解对源域的过拟合。为评估所提方法的有效性，我们在PACS [1]和OfficeHome [2]基准数据集以及实际晶圆半导体缺陷数据集[3]上进行了大量实验。结果表明，RRLD实现了鲁棒且准确的泛化性能。在三个数据集上，我们观察到比现有最优方法提升0.3%至2.3%的改进。