We consider the framework of penalized estimation where the penalty term is given by a real-valued polyhedral gauge, which encompasses methods such as LASSO (and many variants thereof such as the generalized LASSO), SLOPE, OSCAR, PACS and others. Each of these estimators can uncover a different structure or ``pattern'' of the unknown parameter vector. We define a general notion of patterns based on subdifferentials and formalize an approach to measure their complexity. For pattern recovery, we provide a minimal condition for a particular pattern to be detected by the procedure with positive probability, the so-called accessibility condition. Using our approach, we also introduce the stronger noiseless recovery condition. For the LASSO, it is well known that the irrepresentability condition is necessary for pattern recovery with probability larger than $1/2$ and we show that the noiseless recovery plays exactly the same role, thereby extending and unifying the irrepresentability condition of the LASSO to a broad class of penalized estimators. We show that the noiseless recovery condition can be relaxed when turning to thresholded penalized estimators, extending the idea of the thresholded LASSO: we prove that the accessibility condition is already sufficient (and necessary) for sure pattern recovery by thresholded penalized estimation provided that the signal of the pattern is large enough. Throughout the article, we demonstrate how our findings can be interpreted through a geometrical lens.
翻译:我们考虑罚项由实值多面体规范函数给出的惩罚估计框架,涵盖LASSO(及其广义LASSO等众多变体)、SLOPE、OSCAR、PACS等方法。每个估计量都能揭示未知参数向量的不同结构或“模式”。我们基于次微分定义了模式的广义概念,并形式化了度量其复杂度的方法。对于模式恢复,我们给出了特定模式能被过程以正概率检测到的最小条件——即所谓的可及性条件。利用该方法,我们还引入了更强的无噪声恢复条件。对于LASSO,众所周知,不可表示条件是模式恢复概率大于1/2的必要条件,我们证明无噪声恢复恰好发挥相同作用,从而将LASSO的不可表示条件扩展并统一到广泛类别的惩罚估计量中。研究表明,当转向阈值化惩罚估计量时,无噪声恢复条件可以放宽——这一结论延伸了阈值化LASSO的思想:我们证明只要模式信号足够强,可及性条件就足以(且必要)确保阈值化惩罚估计能够可靠恢复模式。本文始终通过几何视角阐释我们的发现。