This paper presents a comprehensive analysis of the growth rate of $H$-consistency bounds (and excess error bounds) for various surrogate losses used in classification. We prove a square-root growth rate near zero for smooth margin-based surrogate losses in binary classification, providing both upper and lower bounds under mild assumptions. This result also translates to excess error bounds. Our lower bound requires weaker conditions than those in previous work for excess error bounds, and our upper bound is entirely novel. Moreover, we extend this analysis to multi-class classification with a series of novel results, demonstrating a universal square-root growth rate for smooth comp-sum and constrained losses, covering common choices for training neural networks in multi-class classification. Given this universal rate, we turn to the question of choosing among different surrogate losses. We first examine how $H$-consistency bounds vary across surrogates based on the number of classes. Next, ignoring constants and focusing on behavior near zero, we identify minimizability gaps as the key differentiating factor in these bounds. Thus, we thoroughly analyze these gaps, to guide surrogate loss selection, covering: comparisons across different comp-sum losses, conditions where gaps become zero, and general conditions leading to small gaps. Additionally, we demonstrate the key role of minimizability gaps in comparing excess error bounds and $H$-consistency bounds.

翻译：本文全面分析了分类任务中各种替代损失的$H$一致性界（以及超额误差界）的增长率。我们证明了在二分类问题中，基于平滑边界的替代损失在接近零处具有平方根增长率，并在温和假设下提供了上下界。该结果同样适用于超额误差界。我们的下界条件比以往关于超额误差界的工作更为宽松，而上界则是全新的。此外，我们将分析扩展至多分类问题，提出一系列创新结果，证明了平滑comp-sum损失和约束损失的通用平方根增长率，涵盖了多分类神经网络训练中的常见选择。基于这一通用增长率，我们进一步探讨不同替代损失的选择问题。首先，我们考察了不同替代损失随类别数量变化的$H$一致性界。其次，忽略常数且聚焦于接近零处的行为，我们将最小化误差缺口识别为这些界的关键区分因素。因此，我们深入分析了这些缺口以指导替代损失选择，涵盖：不同comp-sum损失之间的比较、缺口为零的条件以及导致较小缺口的通用条件。此外，我们证明了最小化误差缺口在比较超额误差界与$H$一致性界中的关键作用。