It is well known that the minimum $\ell_2$-norm solution of the convex LASSO model, say $\mathbf{x}_{\star}$, is a continuous piecewise linear function of the regularization parameter $\lambda$, and its signed sparsity pattern is constant within each linear piece. The current study is an extension of this classic result, proving that the aforementioned properties extend to the min-norm solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, where $\mathbf{y}$ is the observed signal, for a generalization of LASSO termed the scaled generalized minimax concave (sGMC) model. The sGMC model adopts a nonconvex debiased variant of the $\ell_1$-norm as sparse regularizer, but its objective function is overall-convex. Based on the geometric properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we propose an extension of the least angle regression (LARS) algorithm, which iteratively computes the closed-form expression of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ in each linear zone. Under suitable conditions, the proposed algorithm provably obtains the whole solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ within finite iterations. Notably, our proof techniques for establishing continuity and piecewise linearity of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ are novel, and they lead to two side contributions: (a) our proofs establish continuity of the sGMC solution set as a set-valued mapping of $(\mathbf{y},\lambda)$; (b) to prove piecewise linearity and piecewise constant sparsity pattern of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we do not require any assumption that previous work relies on (whereas to prove some additional properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we use a different set of assumptions from previous work).
翻译:众所周知,凸LASSO模型的最小$\ell_2$范数解(记为$\mathbf{x}_{\star}$)是正则化参数$\lambda$的连续分段线性函数,且其符号稀疏模式在每个线性片段内保持恒定。本研究是对这一经典结果的扩展,证明了上述性质可推广到最小范数解映射$\mathbf{x}_{\star}(\mathbf{y},\lambda)$(其中$\mathbf{y}$为观测信号),该映射对应于LASSO的一个推广模型——尺度化广义极小极大凹(sGMC)模型。sGMC模型采用$\ell_1$范数的一种非凸去偏变体作为稀疏正则项,但其目标函数整体是凸的。基于$\mathbf{x}_{\star}(\mathbf{y},\lambda)$的几何性质,我们提出了最小角回归(LARS)算法的一种扩展版本,该算法能迭代计算每个线性区域内$\mathbf{x}_{\star}(\mathbf{y},\lambda)$的闭式表达式。在适当条件下,所提算法可保证在有限次迭代内获得完整的解映射$\mathbf{x}_{\star}(\mathbf{y},\lambda)$。特别值得注意的是,我们证明$\mathbf{x}_{\star}(\mathbf{y},\lambda)$连续性和分段线性性质的技巧是新颖的,并由此产生了两项附加贡献:(a)我们的证明确立了sGMC解集作为$(\mathbf{y},\lambda)$的集值映射的连续性;(b)在证明$\mathbf{x}_{\star}(\mathbf{y},\lambda)$的分段线性性质及分段恒定稀疏模式时,我们无需依赖先前工作中所需的任何假设(而对于证明$\mathbf{x}_{\star}(\mathbf{y},\lambda)$的某些附加性质,我们采用了与先前工作不同的一组假设)。