Domain Generalization (DG) studies the capability of a deep learning model to generalize to out-of-training distributions. In the last decade, literature has been massively filled with training methodologies that claim to obtain more abstract and robust data representations to tackle domain shifts. Recent research has provided a reproducible benchmark for DG, pointing out the effectiveness of naive empirical risk minimization (ERM) over existing algorithms. Nevertheless, researchers persist in using the same outdated feature extractors, and no attention has been given to the effects of different backbones yet. In this paper, we start back to the backbones proposing a comprehensive analysis of their intrinsic generalization capabilities, which so far have been ignored by the research community. We evaluate a wide variety of feature extractors, from standard residual solutions to transformer-based architectures, finding an evident linear correlation between large-scale single-domain classification accuracy and DG capability. Our extensive experimentation shows that by adopting competitive backbones in conjunction with effective data augmentation, plain ERM outperforms recent DG solutions and achieves state-of-the-art accuracy. Moreover, our additional qualitative studies reveal that novel backbones give more similar representations to same-class samples, separating different domains in the feature space. This boost in generalization capabilities leaves marginal room for DG algorithms. It suggests a new paradigm for investigating the problem, placing backbones in the spotlight and encouraging the development of consistent algorithms on top of them. The code is available at https://github.com/PIC4SeR/Back-to-Bones.

翻译：域泛化（Domain Generalization, DG）研究深度学习模型对训练分布外数据进行泛化的能力。过去十年间，大量文献提出了训练方法，声称能够获得更抽象、更鲁棒的数据表示以应对域偏移。近期研究为DG提供了可复现的基准，指出朴素经验风险最小化（ERM）相较于现有算法的有效性。然而，研究者仍沿用相同的陈旧特征提取器，无人关注不同骨干网络的影响。本文回归骨干网络本质，对其内在泛化能力提出全面分析——这一领域迄今被研究界所忽视。我们评估了从标准残差方案到基于Transformer架构的广泛特征提取器，发现大规模单域分类准确率与DG能力之间存在明显的线性相关性。大量实验表明，采用具有竞争力的骨干网络并结合有效数据增强，朴素ERM即可超越近期DG方案，实现最先进准确率。此外，我们的额外定性研究揭示，新型骨干网络能为同类样本生成更相似的表示，同时在特征空间中分离不同域。这种泛化能力的提升使得DG算法仅余边际改进空间。这为问题研究开辟了新范式：将骨干网络置于聚光灯下，鼓励在其基础上开发一致性算法。代码见 https://github.com/PIC4SeR/Back-to-Bones。