We study the connection between multicalibration and boosting for squared error regression. First we prove a useful characterization of multicalibration in terms of a ``swap regret'' like condition on squared error. Using this characterization, we give an exceedingly simple algorithm that can be analyzed both as a boosting algorithm for regression and as a multicalibration algorithm for a class H that makes use only of a standard squared error regression oracle for H. We give a weak learning assumption on H that ensures convergence to Bayes optimality without the need to make any realizability assumptions -- giving us an agnostic boosting algorithm for regression. We then show that our weak learning assumption on H is both necessary and sufficient for multicalibration with respect to H to imply Bayes optimality. We also show that if H satisfies our weak learning condition relative to another class C then multicalibration with respect to H implies multicalibration with respect to C. Finally we investigate the empirical performance of our algorithm experimentally using an open source implementation that we make available. Our code repository can be found at https://github.com/Declancharrison/Level-Set-Boosting.
翻译:摘要:我们研究了多校准与平方误差回归提升方法之间的联系。首先,我们基于平方误差的"交换遗憾"条件,证明了多校准的一个有用表征。利用该表征,我们提出了一种极其简单的算法,该算法既可被分析为回归提升算法,也可被分析为仅需使用H的标准平方误差回归预言机的多校准算法。我们给出了关于H的弱学习假设,该假设无需任何可实现性假设即可确保收敛到贝叶斯最优性——从而得到一种回归的不可知提升算法。随后证明,关于H的弱学习假设既是多校准蕴含贝叶斯最优性的必要条件也是充分条件。我们还发现,若H相对于另一类别C满足弱学习条件,则关于H的多校准可蕴含关于C的多校准。最后,我们利用开源实现进行了实验性算法性能研究,相关代码仓库可见于https://github.com/Declancharrison/Level-Set-Boosting。