The theory of boosting provides a computational framework for aggregating approximate weak learning algorithms, which perform marginally better than a random predictor, into an accurate strong learner. In the realizable case, the success of the boosting approach is underscored by a remarkable fact that the resultant sample complexity matches that of a computationally demanding alternative, namely Empirical Risk Minimization (ERM). This in particular implies that the realizable boosting methodology has the potential to offer computational relief without compromising on sample efficiency. Despite recent progress, in agnostic boosting, where assumptions on the conditional distribution of labels given feature descriptions are absent, ERM outstrips the agnostic boosting methodology in being quadratically more sample efficient than all known agnostic boosting algorithms. In this paper, we make progress on closing this gap, and give a substantially more sample efficient agnostic boosting algorithm than those known, without compromising on the computational (or oracle) complexity. A key feature of our algorithm is that it leverages the ability to reuse samples across multiple rounds of boosting, while guaranteeing a generalization error strictly better than those obtained by blackbox applications of uniform convergence arguments. We also apply our approach to other previously studied learning problems, including boosting for reinforcement learning, and demonstrate improved results.

翻译：提升理论为聚合近似弱学习算法提供了一个计算框架，这些算法的性能略优于随机预测器，从而形成精确的强学习器。在可实现的情况下，提升方法的成功突显了一个显著事实：其产生的样本复杂度与计算要求较高的替代方法——即经验风险最小化（ERM）——相匹配。这尤其意味着，可实现提升方法有潜力在不牺牲样本效率的前提下提供计算上的缓解。尽管最近取得了进展，但在无知提升中（即给定特征描述时，不对标签的条件分布作任何假设），ERM在样本效率上超越了无知提升方法，其样本效率比所有已知的无知提升算法高出二次方量级。在本文中，我们在缩小这一差距方面取得了进展，提出了一种比已知算法样本效率显著更高的无知提升算法，且未牺牲计算（或预言机）复杂度。我们算法的一个关键特点是，它利用了在提升的多轮迭代中重复使用样本的能力，同时保证泛化误差严格优于通过黑箱应用一致收敛论证所得到的结果。我们还将我们的方法应用于其他先前研究过的学习问题，包括强化学习中的提升，并展示了改进的结果。