Pathwise coordinate descent algorithms have been used to compute entire solution paths for lasso and other penalized regression problems quickly with great success. They improve upon cold start algorithms by solving the problems that make up the solution path sequentially for an ordered set of tuning parameter values, instead of solving each problem separately. However, extending pathwise coordinate descent algorithms to more the general bridge or power family of $\ell_q$ penalties is challenging. Faster algorithms for computing solution paths for these penalties are needed because $\ell_q$ penalized regression problems can be nonconvex and especially burdensome to solve. In this paper, we show that a reparameterization of $\ell_q$ penalized regression problems is more amenable to pathwise coordinate descent algorithms. This allows us to improve computation of the mode-thresholding function for $\ell_q$ penalized regression problems in practice and introduce two separate pathwise algorithms. We show that either pathwise algorithm is faster than the corresponding cold-start alternative, and demonstrate that different pathwise algorithms may be more likely to reach better solutions.
翻译:路径坐标下降算法已被成功用于快速计算套索及其他惩罚回归问题的完整解路径。与冷启动算法相比,该方法通过按顺序求解一组有序调谐参数值对应的子问题来构成解路径,而非分别独立求解每个问题,从而实现了性能提升。然而,将路径坐标下降算法推广至更一般的桥罚或$\ell_q$幂罚族面临挑战。由于$\ell_q$惩罚回归问题可能非凸且求解负担尤为沉重,亟需更高效的解路径计算算法。本文证明,对$\ell_q$惩罚回归问题重新参数化能更好地适应路径坐标下降算法。这使得我们能够实际改进$\ell_q$惩罚回归问题中模态阈值函数的计算,并提出两种不同的路径算法。实验表明,两种路径算法均比对应的冷启动替代方案更快,同时验证了不同路径算法更有可能收敛到更优解。