In this study, an exact coordinate descent algorithm is developed for high-dimensional Huber regression regularized with an elastic net penalty. Unlike existing gradient descent or coordinate descent-type methods, this algorithm remains effective even when the Hessian becomes ill-conditioned due to high correlations between covariates drawn from heavy-tailed distributions. For each coordinate, marginal increments arise solely from inlier observations, while the derivatives remain monotonically increasing over a grid constructed from the partial residuals. Building on conventional coordinate descent frameworks, adaptive variable screening rules are proposed to selectively determine which variables to update at each iteration, thereby accelerating convergence. The convergence of the proposed algorithm is formally analyzed, and practical computational strategies are presented to speed up its execution. These enhancements ensure that the algorithm operates rapidly and stably even under challenging scenarios. Extensive simulation studies involving heavy-tailed noise and highly correlated predictors, along with a real-world data application, demonstrate both the practical efficiency of this method and the benefits of the computational enhancements.
翻译:本研究针对弹性网惩罚正则化的高维Huber回归,提出了一种精确坐标下降算法。与现有的梯度下降或坐标下降类方法不同,即使因重尾分布协变量间的高度相关性导致海森矩阵病态时,该算法仍能保持有效性。对于每个坐标,边际增量仅由正常观测值产生,而导数在基于部分残差构建的网格上保持单调递增。基于传统坐标下降框架,本文提出自适应变量筛选规则,以选择性地确定每次迭代中需要更新的变量,从而加速收敛。文中对该算法的收敛性进行了正式分析,并给出了实用的计算策略以提高执行速度。这些改进确保算法即使在具有挑战性的场景下也能快速稳定运行。基于重尾噪声与高度相关预测变量的广泛模拟研究,以及实际数据的应用案例,均证明了该方法在实践中具有良好的效率以及计算改进带来的实际效益。