In boosting, we aim to leverage multiple weak learners to produce a strong learner. At the center of this paradigm lies the concept of building the strong learner as a voting classifier, which outputs a weighted majority vote of the weak learners. While many successful boosting algorithms, such as the iconic AdaBoost, produce voting classifiers, their theoretical performance has long remained sub-optimal: the best known bounds on the number of training examples necessary for a voting classifier to obtain a given accuracy has so far always contained at least two logarithmic factors above what is known to be achievable by general weak-to-strong learners. In this work, we break this barrier by proposing a randomized boosting algorithm that outputs voting classifiers whose generalization error contains a single logarithmic dependency on the sample size. We obtain this result by building a general framework that extends sample compression methods to support randomized learning algorithms based on sub-sampling.

翻译：在提升算法中，我们旨在利用多个弱学习器来产生一个强学习器。该范式的核心在于将强学习器构建为投票分类器，即输出弱学习器的加权多数投票结果。尽管许多成功的提升算法（如经典的AdaBoost）能够产生投票分类器，但其理论性能长期以来一直未达最优：目前已知的、投票分类器在达到给定精度时所需训练样本数的最佳上界，通常至少包含两个对数因子，超过了通用弱到强学习器所能达到的水平。在本研究中，我们通过提出一种随机提升算法打破了这一壁垒，该算法输出的投票分类器的泛化误差仅与样本量呈单对数依赖关系。我们通过构建一个通用框架实现此结果，该框架将样本压缩方法扩展到支持基于子采样的随机学习算法。