The one-inclusion graph algorithm of Haussler, Littlestone, and Warmuth achieves an optimal in-expectation risk bound in the standard PAC classification setup. In one of the first COLT open problems, Warmuth conjectured that this prediction strategy always implies an optimal high probability bound on the risk, and hence is also an optimal PAC algorithm. We refute this conjecture in the strongest sense: for any practically interesting Vapnik-Chervonenkis class, we provide an in-expectation optimal one-inclusion graph algorithm whose high probability risk bound cannot go beyond that implied by Markov's inequality. Our construction of these poorly performing one-inclusion graph algorithms uses Varshamov-Tenengolts error correcting codes. Our negative result has several implications. First, it shows that the same poor high-probability performance is inherited by several recent prediction strategies based on generalizations of the one-inclusion graph algorithm. Second, our analysis shows yet another statistical problem that enjoys an estimator that is provably optimal in expectation via a leave-one-out argument, but fails in the high-probability regime. This discrepancy occurs despite the boundedness of the binary loss for which arguments based on concentration inequalities often provide sharp high probability risk bounds.

翻译：Haussler、Littlestone 和 Warmuth 的单入式图表算法在标准 PAC 分类设置中实现了最佳的预期风险。 在第一个COLT公开的问题之一中, Warmuth 推测,这种预测战略总是意味着对风险的最佳高概率,因此也是最佳的 PAC 算法。 我们最强烈地驳斥了这种推测:对于任何实际令人感兴趣的Vapnik-Chervonenkis 类的Vapnik-Chervonenkis 预测,我们提供了一种最佳的预入式单入式图表算法,其高度概率风险无法超过Markov 不平等所隐含的风险。我们在构建这些执行不良的单入式图表算法时,使用了Varshamov-Tenngolt 错误校正代码。 我们的负面结果产生了若干影响。 首先,我们从最近的一些基于一入式图形算法的概括性预测战略中继承了同样的高概率性表现。 其次,我们的分析表明另一个统计问题,这个估计者认为, 最理想的预想中, 最理想的一入局性是高偏差偏差性 。