To improve model generalization, model designers often restrict the features that their models use, either implicitly or explicitly. In this work, we explore the design space of leveraging such feature priors by viewing them as distinct perspectives on the data. Specifically, we find that models trained with diverse sets of feature priors have less overlapping failure modes, and can thus be combined more effectively. Moreover, we demonstrate that jointly training such models on additional (unlabeled) data allows them to correct each other's mistakes, which, in turn, leads to better generalization and resilience to spurious correlations. Code available at https://github.com/MadryLab/copriors.

翻译：为了改进模型的概括化,模型设计师往往以隐含或明确的方式限制其模型使用的特征。在这项工作中,我们探索利用这些特征前科的设计空间,将它们视为数据的不同视角。具体地说,我们发现,经过不同特征前科培训的模型的重叠性失败模式较少,因此可以更有效地结合。此外,我们证明,在额外(未贴标签的)数据上联合培训这些模型,可以使他们纠正彼此的错误,这反过来又会改善对虚假关联的概括性和复原力。代码见https://github.com/MadryLab/coprirors。