We investigate the asymptotic behaviour of gradient boosting algorithms when the learning rate converges to zero and the number of iterations is rescaled accordingly. We mostly consider L2-boosting for regression with linear base learner as studied in B{\"u}hlmann and Yu (2003) and analyze also a stochastic version of the model where subsampling is used at each step (Friedman 2002). We prove a deterministic limit in the vanishing learning rate asymptotic and characterize the limit as the unique solution of a linear differential equation in an infinite dimensional function space. Besides, the training and test error of the limiting procedure are thoroughly analyzed. We finally illustrate and discuss our result on a simple numerical experiment where the linear L2-boosting operator is interpreted as a smoothed projection and time is related to its number of degrees of freedom.

翻译：当学习率接近零时,我们调查梯度助推算法的无症状行为,并相应调整迭代次数。我们大多考虑B{u}hlmann和Yu(2003年)所研究的线性基本学习者回归的L2-加速率,并分析每步使用子抽样的模型的随机化版本(Friedman,2002年)。我们证明,在消失的学习率无症状中,我们是一个决定性的极限,并将极限定性为无限维功能空间线性差异方程式的独特解决方案。此外,对限制程序的训练和测试错误进行了彻底分析。我们最后用简单的数字实验来说明和讨论我们的结果,即线性L2-接轨操作者被解释为一个平稳的预测,时间与其自由度的数量相关。