This paper deals with tactics for fast computation in least squares regression in high dimensions. These tactics include: (a) the majorization-minimization (MM) principle, (b) smoothing by Moreau envelopes, and (c) the proximal distance principle for constrained estimation. In iteratively reweighted least squares, the MM principle can create a surrogate function that trades case weights for adjusted responses. Reduction to ordinary least squares then permits the reuse of the Gram matrix and its Cholesky decomposition across iterations. This tactic is pertinent to estimation in L2E regression and generalized linear models. For problems such as quantile regression, non-smooth terms of an objective function can be replaced by their Moreau envelope approximations and majorized by spherical quadratics. Finally, penalized regression with distance-to-set penalties also benefits from this perspective. Our numerical experiments validate the speed and utility of deweighting and Moreau envelope approximations. Julia software implementing these experiments is available on our web page.

翻译：本文探讨了高维最小二乘回归中快速计算的优化策略。这些策略包括：(a) 极大极小化（MM）原理，(b) 基于Moreau包络的光滑化，以及(c) 用于约束估计的近端距离原理。在迭代重加权最小二乘法中，MM原理可构造一个替代函数，将案例权重转化为调整响应。通过简化为普通最小二乘法，可在迭代过程中复用Gram矩阵及其Cholesky分解。该策略适用于L2E回归与广义线性模型的估计。对于分位数回归等问题，目标函数中的非光滑项可被其Moreau包络逼近替代，并通过球面二次函数进行优化。此外，基于距离惩罚的惩罚回归亦能受益于该框架。数值实验验证了去权重与Moreau包络逼近的速度与实用性。实现这些实验的Julia软件可在我们的网页获取。