AdaBoost is a classic boosting algorithm for combining multiple inaccurate classifiers produced by a weak learner, to produce a strong learner with arbitrarily high accuracy when given enough training data. Determining the optimal number of samples necessary to obtain a given accuracy of the strong learner, is a basic learning theoretic question. Larsen and Ritzert (NeurIPS'22) recently presented the first provably optimal weak-to-strong learner. However, their algorithm is somewhat complicated and it remains an intriguing question whether the prototypical boosting algorithm AdaBoost also makes optimal use of training samples. In this work, we answer this question in the negative. Concretely, we show that the sample complexity of AdaBoost, and other classic variations thereof, are sub-optimal by at least one logarithmic factor in the desired accuracy of the strong learner.

翻译：AdaBoost是一种经典的集成学习算法，其通过组合弱学习器生成的多个不精确分类器，在给定足够训练数据时，能够构建出任意高精度的强学习器。确定获取强学习器特定精度所需的最优样本数量，是学习理论中的基本问题。Larsen与Ritzert（NeurIPS'22）近期提出了首个被证明为最优的弱到强学习器。然而，该算法相对复杂，而原型提升算法AdaBoost是否也能最优地利用训练样本仍是一个引人关注的问题。本研究对此问题给出了否定回答。具体而言，我们证明AdaBoost及其经典变体的样本复杂度在强学习器期望精度上至少存在一个对数因子的次优性。