Developing an optimal PAC learning algorithm in the realizable setting, where empirical risk minimization (ERM) is suboptimal, was a major open problem in learning theory for decades. The problem was finally resolved by Hanneke a few years ago. Unfortunately, Hanneke's algorithm is quite complex as it returns the majority vote of many ERM classifiers that are trained on carefully selected subsets of the data. It is thus a natural goal to determine the simplest algorithm that is optimal. In this work we study the arguably simplest algorithm that could be optimal: returning the majority vote of three ERM classifiers. We show that this algorithm achieves the optimal in-expectation bound on its error which is provably unattainable by a single ERM classifier. Furthermore, we prove a near-optimal high-probability bound on this algorithm's error. We conjecture that a better analysis will prove that this algorithm is in fact optimal in the high-probability regime.

翻译：在可实现设定下，经验风险最小化（ERM）次优时，设计最优的PAC学习算法是学习理论中数十年未解的重大开放问题。该问题最终由Hanneke在数年前解决。遗憾的是，Hanneke算法相当复杂——它返回多个在精心挑选的数据子集上训练的ERM分类器的多数投票结果。因此，确定最简单的可达到最优性的算法成为自然目标。本研究探讨了可能是最简单的候选最优算法：返回三个ERM分类器的多数投票结果。我们证明该算法实现了期望误差的最优界，且单ERM分类器无法达到该界。进一步，我们推导了该算法误差的近最优高概率界。我们推测，更精细的分析将证明该算法在高概率意义下确实是最优的。