The elastic net combines lasso and ridge regression to fuse the sparsity property of lasso with the grouping property of ridge regression. The connections between ridge regression and gradient descent and between lasso and forward stagewise regression have previously been shown. Similar to how the elastic net generalizes lasso and ridge regression, we introduce elastic gradient descent, a generalization of gradient descent and forward stagewise regression. We theoretically analyze elastic gradient descent and compare it to the elastic net and forward stagewise regression. Parts of the analysis are based on elastic gradient flow, a piecewise analytical construction, obtained for elastic gradient descent with infinitesimal step size. We also compare elastic gradient descent to the elastic net on real and simulated data and show that it provides similar solution paths, but is several orders of magnitude faster. Compared to forward stagewise regression, elastic gradient descent selects a model that, although still sparse, provides considerably lower prediction and estimation errors.

翻译：弹性网结合了LASSO与岭回归，既保留了LASSO的稀疏性特征，又融合了岭回归的分组特性。已有研究证明了岭回归与梯度下降、LASSO与前向分段回归之间的关联。与弹性网作为LASSO和岭回归的推广类似，本文提出弹性梯度下降——一种梯度下降与前向分段回归的泛化方法。我们从理论上分析了弹性梯度下降，并将其与弹性网及前向分段回归进行了比较。部分分析基于弹性梯度流（一种针对无穷小步长弹性梯度下降的分段解析构造）。通过在真实与模拟数据上将弹性梯度下降与弹性网进行对比，我们证明该方法能产生相似的解路径，但速度快数个数量级。相较于前向分段回归，弹性梯度下降选择的模型虽然仍保持稀疏性，但其预测误差和估计误差显著降低。