Pathwise coordinate descent algorithms have been used to compute entire solution paths for lasso and other penalized regression problems quickly with great success. They improve upon cold start algorithms by solving the problems that make up the solution path sequentially for an ordered set of tuning parameter values, instead of solving each problem separately. However, extending pathwise coordinate descent algorithms to more the general bridge or power family of $\ell_q$ penalties is challenging. Faster algorithms for computing solution paths for these penalties are needed because $\ell_q$ penalized regression problems can be nonconvex and especially burdensome to solve. In this paper, we show that a reparameterization of $\ell_q$ penalized regression problems is more amenable to pathwise coordinate descent algorithms. This allows us to improve computation of the mode-thresholding function for $\ell_q$ penalized regression problems in practice and introduce two separate pathwise algorithms. We show that either pathwise algorithm is faster than the corresponding cold-start alternative, and demonstrate that different pathwise algorithms may be more likely to reach better solutions.
翻译:路径坐标下降算法已被成功用于快速计算lasso及其他惩罚回归问题的完整解路径。与冷启动算法不同,这些算法通过按顺序求解参数调优值有序集合对应的子问题来改进计算效率,而非独立求解每个问题。然而,将路径坐标下降算法推广至更一般的桥梁族或$\ell_q$惩罚函数族颇具挑战性。由于$\ell_q$惩罚回归问题可能非凸且求解过程尤为繁重,亟需更快速的算法用于计算这些惩罚的解路径。本文表明,$\ell_q$惩罚回归问题的重参数化更适用于路径坐标下降算法。这使我们能够实际改进$\ell_q$惩罚回归问题的模阈值函数计算,并引入两种独立的路径算法。我们证明,两种路径算法的计算速度均优于对应的冷启动替代方案,并验证不同路径算法更有可能获得更优解。