The elastic net combines lasso and ridge regression to fuse the sparsity property of lasso with the grouping property of ridge regression. The connections between ridge regression and gradient descent and between lasso and forward stagewise regression have previously been shown. Similarity to how the elastic net generalizes lasso and ridge regression, we introduce elastic gradient descent, a generalization of gradient descent and forward stagewise regression. We compare elastic gradient descent to the elastic net on real and simulated data, and show that it provides similar solution paths, but is of several orders of magnitude faster. Compared to forward stagewise regression, elastic gradient descent selects a model that, although still sparse, provides considerable lower prediction and estimation errors. We also investigate the case of infinitesimal step size, obtaining a piecewise analytical solution we refer to as elastic gradient flow.

翻译：弹性网络结合了套索回归与岭回归，融合了套索的稀疏性特性和岭回归的分组特性。已有研究揭示了岭回归与梯度下降之间、套索回归与前向分段回归之间的内在联系。类似于弹性网络对套索和岭回归的泛化过程，我们提出了弹性梯度下降方法——这是对梯度下降法和前向分段回归的一种推广。通过在实际数据与模拟数据上的对比实验表明，弹性梯度下降能生成与弹性网络相似的解路径，但计算速度提升数个数量级。相较于前向分段回归，弹性梯度下降选出的模型在保持稀疏性的同时，显著降低了预测误差和估计误差。我们还探讨了极小步长情形，获得了一种分段解析解，并将其称为弹性梯度流。