Boosting provides a practical and provably effective framework for constructing accurate learning algorithms from inaccurate rules of thumb. It extends the promise of sample-efficient learning to settings where direct Empirical Risk Minimization (ERM) may not be implementable efficiently. In the realizable setting, boosting is known to offer this computational reprieve without compromising on sample efficiency. However, in the agnostic case, existing boosting algorithms fall short of achieving the optimal sample complexity. This paper highlights an unexpected and previously unexplored avenue of improvement: unlabeled samples. We design a computationally efficient agnostic boosting algorithm that matches the sample complexity of ERM, given polynomially many additional unlabeled samples. In fact, we show that the total number of samples needed, unlabeled and labeled inclusive, is never more than that for the best known agnostic boosting algorithm -- so this result is never worse -- while only a vanishing fraction of these need to be labeled for the algorithm to succeed. This is particularly fortuitous for learning-theoretic applications of agnostic boosting, which often take place in the distribution-specific setting, where unlabeled samples can be availed for free. We detail other applications of this result in reinforcement learning.

翻译：提升为从粗略经验法则构建精确学习算法提供了一个实用且可证明有效的框架。它将样本高效学习的承诺扩展到了直接经验风险最小化可能无法高效实现的场景。在可实现性设定下，已知提升能够在不牺牲样本效率的前提下提供这种计算上的缓解。然而，在不可知情形中，现有的提升算法未能达到最优样本复杂度。本文揭示了一条出乎意料且此前未被探索的改进途径：未标记样本。我们设计了一种计算高效的不可知提升算法，在给定多项式数量额外未标记样本的条件下，其样本复杂度与经验风险最小化相匹配。事实上，我们证明所需的总样本数（包括未标记和标记样本）从未超过已知最佳不可知提升算法所需的数量——因此该结果绝不会更差——而其中仅需极小一部分样本被标记即可保证算法成功。这对于不可知提升的学习理论应用尤为有利，因为这些应用通常发生在分布特定的设定中，其中未标记样本可以免费获取。我们详细阐述了该结果在强化学习中的其他应用。