This paper introduces an iterative algorithm for training nonparametric additive models that enjoys favorable memory storage and computational requirements. The algorithm can be viewed as the functional counterpart of stochastic gradient descent, applied to the coefficients of a truncated basis expansion of the component functions. We show that the resulting estimator satisfies an oracle inequality that allows for model mis-specification. In the well-specified setting, by choosing the learning rate carefully across three distinct stages of training, we demonstrate that its risk is minimax optimal in terms of the dependence on both the dimensionality of the data and the size of the training sample. Unlike past work, we also provide polynomial convergence rates even when the covariates do not have full support on their domain.

翻译：本文提出一种用于训练非参数可加模型的迭代算法，该算法在内存存储和计算需求方面具有显著优势。该算法可视为随机梯度下降在函数空间中的对应方法，应用于分量函数截断基展开的系数上。我们证明所得估计量满足允许模型误设的oracle不等式。在模型设定正确的条件下，通过精心选择训练三个不同阶段的学习率，我们证明其风险在数据维度和训练样本量的依赖关系上达到极小极大最优。与以往研究不同，即使协变量在其定义域上不具备完全支撑，我们仍能给出多项式收敛速率。