In statistical learning theory, determining the sample complexity of realizable binary classification for VC classes was a long-standing open problem. The results of Simon and Hanneke established sharp upper bounds in this setting. However, the reliance of their argument on the uniform convergence principle limits its applicability to more general learning settings such as multiclass classification. In this paper, we address this issue by providing optimal high probability risk bounds through a framework that surpasses the limitations of uniform convergence arguments. Our framework converts the leave-one-out error of permutation invariant predictors into high probability risk bounds. As an application, by adapting the one-inclusion graph algorithm of Haussler, Littlestone, and Warmuth, we propose an algorithm that achieves an optimal PAC bound for binary classification. Specifically, our result shows that certain aggregations of one-inclusion graph algorithms are optimal, addressing a variant of a classic question posed by Warmuth. We further instantiate our framework in three settings where uniform convergence is provably suboptimal. For multiclass classification, we prove an optimal risk bound that scales with the one-inclusion hypergraph density of the class, addressing the suboptimality of the analysis of Daniely and Shalev-Shwartz. For partial hypothesis classification, we determine the optimal sample complexity bound, resolving a question posed by Alon, Hanneke, Holzman, and Moran. For realizable bounded regression with absolute loss, we derive an optimal risk bound that relies on a modified version of the scale-sensitive dimension, refining the results of Bartlett and Long. Our rates surpass standard uniform convergence-based results due to the smaller complexity measure in our risk bound.

翻译：在统计学习理论中，确定VC类的可实现二元分类的样本复杂度一直是一个长期悬而未决的问题。Simon和Hanneke的结果建立了该情形下的尖锐上界。然而，他们的论证依赖于统一收敛原理，这限制了其在更一般的学习设置（如多类分类）中的适用性。本文通过提供一个超越统一收敛论证局限性的框架，推导出最优的高概率风险界，从而解决了这一问题。我们的框架将置换不变预测器的留一误差转化为高概率风险界。作为一个应用，通过调整Haussler、Littlestone和Warmuth的一包含图算法，我们提出了一种在二元分类中实现最优PAC界的算法。具体而言，我们的结果表明，一包含图算法的某些聚合是最优的，从而解决了Warmuth提出的一个经典问题的变体。我们进一步在三个统一收敛被证明是次优的设置中实例化了该框架。对于多类分类，我们证明了一个与类的一包含超图密度成正比的最优风险界，解决了Daniely和Shalev-Shwartz分析中的次优性。对于部分假设分类，我们确定了最优样本复杂度界，解决了Alon、Hanneke、Holzman和Moran提出的一个问题。对于具有绝对损失的可实现有界回归，我们推导了一个依赖于尺度敏感维度修正版本的最优风险界，改进了Bartlett和Long的结果。由于我们的风险界中采用了更小的复杂度度量，我们的速率超过了基于标准统一收敛的结果。