In prediction tasks, there exist features that are related to the label in the same way across different settings for that task; these are semantic features or semantics. Features with varying relationships to the label are nuisances. For example, in detecting cows from natural images, the shape of the head is semantic but because images of cows often have grass backgrounds but not always, the background is a nuisance. Models that exploit nuisance-label relationships face performance degradation when these relationships change. Building models robust to such changes requires additional knowledge beyond samples of the features and labels. For example, existing work uses annotations of nuisances or assumes ERM-trained models depend on nuisances. Approaches to integrate new kinds of additional knowledge enlarge the settings where robust models can be built. We develop an approach to use knowledge about the semantics by corrupting them in data, and then using the corrupted data to produce models which identify correlations between nuisances and the label. Once these correlations are identified, they can be used to adjust for where nuisances drive predictions. We study semantic corruptions in powering different spurious-correlation avoiding methods on multiple out-of-distribution (OOD) tasks like classifying waterbirds, natural language inference (NLI), and detecting cardiomegaly in chest X-rays.

翻译：在预测任务中，存在某些特征在不同任务场景下与标签保持相同关联方式，这类特征称为语义特征或语义信息。而与标签关联关系不稳定的特征则属于干扰因素。例如，在自然图像中检测奶牛时，头部形状属于语义特征；但由于奶牛图像常以草地为背景（并非总是如此），背景信息便构成干扰因素。当干扰因素与标签的关联关系发生变化时，依赖这种关系的模型会出现性能下降。要构建对此类变化具有鲁棒性的模型，仅依靠特征和标签样本是不够的，还需要额外先验知识。现有研究通常采用干扰因素标注数据，或假设基于经验风险最小化（ERM）训练的模型会依赖干扰因素。整合新型先验知识的方法能够拓展可构建鲁棒模型的场景范围。本文提出一种利用语义知识的方法：通过在数据中破坏语义信息，进而利用破坏后的数据构建模型以识别干扰因素与标签之间的相关性。一旦识别出这些相关性，即可据此调整由干扰因素主导的预测结果。我们通过语义破坏技术，在多种分布外（OOD）任务中验证了不同避免伪相关性方法的有效性，包括水鸟分类、自然语言推理（NLI）以及胸部X光片中的心脏肥大检测。