The choice of loss function in classification involves a fundamental trade-off: smooth losses (like Cross-Entropy) enable fast optimization rates but yield slow square-root consistency bounds, while piecewise-linear losses (like Hinge) offer fast linear consistency rates but suffer from non-differentiability. We propose Linear-Core (LC) Surrogates, a new family of convex loss functions that resolve this tension by stitching a linear core to a smooth tail. We prove that these surrogates are differentiable everywhere while retaining strict linear $H$-consistency bounds, effectively combining the optimization benefits of smoothness with the statistical efficiency of margin-based losses. In the structured prediction setting, we show that this smoothness unlocks a massive computational and energy advantage: it allows for an unbiased stochastic gradient estimator that bypasses the quadratic complexity $O(|\mathscr{Y}|^2)$ of exact inference (e.g., Viterbi). Empirically, our method achieves a 23$\times$ speedup over Structured SVMs on large-vocabulary sequence tagging tasks and demonstrates superior robustness to instance-dependent label noise, outperforming Cross-Entropy by 2.6% on corrupted CIFAR-10.
翻译:损失函数的选择在分类中涉及一个基本权衡:光滑损失(如交叉熵)能够实现快速的优化速率,但产生缓慢的平方根一致性边界;而分段线性损失(如合页损失)提供快速的线性一致性速率,却面临不可微性问题。我们提出线性核心(Linear-Core, LC)代理函数,这是一类新的凸损失函数族,通过将线性核心与光滑尾部拼接来解决这一矛盾。我们证明这些代理函数在保持严格线性$H$-一致性边界的同时处处可微,有效结合了光滑性的优化优势与基于间隔损失的统计效率。在结构化预测场景中,我们展示了这种光滑性带来了巨大的计算和能量优势:它允许一种无偏随机梯度估计器,绕过了精确推理(如维特比算法)的二次复杂度$O(|\mathscr{Y}|^2)$。实验上,我们的方法在大词汇量序列标注任务上比结构化支持向量机实现了23倍的加速,并在对实例相关标签噪声表现出优越的鲁棒性,在受损CIFAR-10数据集上比交叉熵高出2.6%。