Deep neural networks are highly performant, but might base their decision on spurious or background features that co-occur with certain classes, which can hurt generalization. To mitigate this issue, the usage of 'model guidance' has gained popularity recently: for this, models are guided to be "right for the right reasons" by regularizing the models' explanations to highlight the right features. Experimental validation of these approaches has thus far however been limited to relatively simple and / or synthetic datasets. To gain a better understanding of which model-guiding approaches actually transfer to more challenging real-world datasets, in this work we conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets, and show that model guidance can sometimes even improve model performance. In this context, we further propose a novel energy loss, show its effectiveness in directing the model to focus on object features. We also show that these gains can be achieved even with a small fraction (e.g. 1%) of bounding box annotations, highlighting the cost effectiveness of this approach. Lastly, we show that this approach can also improve generalization under distribution shifts. Code will be made available.

翻译：深度神经网络虽然性能卓越，但可能基于与某些类别共存的虚假特征或背景特征做出决策，这会损害模型的泛化能力。为解决这一问题，"模型指导"方法近期备受关注：通过正则化模型解释以突出正确特征，从而引导模型"以正确理由得出正确结论"。然而，目前对这些方法的实验验证仍局限于相对简单或合成的数据集。为深入探究哪些模型指导方法能真正迁移至更具挑战性的真实数据集，本研究在PASCAL VOC 2007和MS COCO 2014数据集上，针对多种损失函数、归因方法、模型及"指导深度"进行了全面评估，结果表明模型指导有时甚至能提升模型性能。在此背景下，我们进一步提出一种新型能量损失函数，并证明其在引导模型聚焦物体特征方面的有效性。实验还表明，即使仅使用少量边界框标注（如1%），也能获得性能提升，凸显了该方法的经济性。最后，我们证明该方法在分布偏移下也能改善泛化能力。相关代码将公开。