Common explanations for shortcut learning assume that the shortcut improves prediction under the training distribution but not in the test distribution. Thus, models trained via the typical gradient-based optimization of cross-entropy, which we call default-ERM, utilize the shortcut. However, even when the stable feature determines the label in the training distribution and the shortcut does not provide any additional information, like in perception tasks, default-ERM still exhibits shortcut learning. Why are such solutions preferred when the loss for default-ERM can be driven to zero using the stable feature alone? By studying a linear perception task, we show that default-ERM's preference for maximizing the margin leads to models that depend more on the shortcut than the stable feature, even without overparameterization. This insight suggests that default-ERM's implicit inductive bias towards max-margin is unsuitable for perception tasks. Instead, we develop an inductive bias toward uniform margins and show that this bias guarantees dependence only on the perfect stable feature in the linear perception task. We develop loss functions that encourage uniform-margin solutions, called margin control (MARG-CTRL). MARG-CTRL mitigates shortcut learning on a variety of vision and language tasks, showing that better inductive biases can remove the need for expensive two-stage shortcut-mitigating methods in perception tasks.

翻译：关于捷径学习的常见解释认为，捷径能提升训练分布下的预测性能，但无法在测试分布下起作用。因此，通过典型的基于梯度的交叉熵优化（我们称之为默认经验风险最小化，Default-ERM）训练的模型会利用捷径。然而，即使在训练分布中稳定特征能完全确定标签，而捷径不提供任何额外信息（例如在感知任务中），默认ERM仍然表现出捷径学习。为什么当仅使用稳定特征就能将默认ERM的损失降至零时，模型仍倾向于选择此类解？通过研究一个线性感知任务，我们证明默认ERM倾向于最大化间隔（margin），这会导致模型更依赖捷径而非稳定特征，即便在未过参数化的情况下也是如此。这一见解表明，默认ERM隐含的“最大间隔”归纳偏置并不适用于感知任务。为此，我们开发了一种向“均匀间隔”的归纳偏置，并证明该偏置能保证在感知任务中模型仅依赖完美的稳定特征。我们设计了鼓励均匀间隔解的损失函数，称为间隔控制（MARG-CTRL）。MARG-CTRL在多种视觉和语言任务中缓解了捷径学习，表明更优的归纳偏置可以消除感知任务中昂贵的两阶段捷径缓解方法的需求。