Given $n$ noisy samples with $p$ dimensions, where $n \ll p$, we show that the multi-step thresholding procedure based on the Lasso -- we call it the {\it Thresholded Lasso}, can accurately estimate a sparse vector $\beta \in {\mathbb R}^p$ in a linear model $Y = X \beta + \epsilon$, where $X_{n \times p}$ is a design matrix normalized to have column $\ell_2$-norm $\sqrt{n}$, and $\epsilon \sim N(0, \sigma^2 I_n)$. We show that under the restricted eigenvalue (RE) condition, it is possible to achieve the $\ell_2$ loss within a logarithmic factor of the ideal mean square error one would achieve with an {\em oracle } while selecting a sufficiently sparse model -- hence achieving {\it sparse oracle inequalities}; the oracle would supply perfect information about which coordinates are non-zero and which are above the noise level. We also show for the Gauss-Dantzig selector (Cand\`{e}s-Tao 07), if $X$ obeys a uniform uncertainty principle, one will achieve the sparse oracle inequalities as above, while allowing at most $s_0$ irrelevant variables in the model in the worst case, where $s_0 \leq s$ is the smallest integer such that for $\lambda = \sqrt{2 \log p/n}$, $\sum_{i=1}^p \min(\beta_i^2, \lambda^2 \sigma^2) \leq s_0 \lambda^2 \sigma^2$. Our simulation results on the Thresholded Lasso match our theoretical analysis excellently.

翻译：给定$n$个含噪样本，维度$p$满足$n \ll p$，我们证明基于Lasso的多步阈值化方法（称之为阈值Lasso）能够在线性模型$Y = X \beta + \epsilon$中准确估计稀疏向量$\beta \in {\mathbb R}^p$。其中设计矩阵$X_{n \times p}$经过归一化处理，其列向量的$\ell_2$范数为$\sqrt{n}$，且$\epsilon \sim N(0, \sigma^2 I_n)$。我们证明，在限制特征值（RE）条件下，该方法能在选取充分稀疏模型的同时，使$\ell_2$损失达到理想均方误差的对数因子范围内（而理想均方误差仅在获得{\em Oracle}时才能实现），从而获得{\em 稀疏Oracle不等式}。若得Oracle帮助，研究人员可准确知晓哪些坐标非零、哪些低于噪声水平。同时证明，对于Gauss-Dantzig选择器（Cand\`{e}s-Tao 2007），若矩阵$X$满足一致不确定性原理，则该选择器将同样实现上述稀疏Oracle不等式，且在最坏情况下模型最多允许包含$s_0$个无关变量。这里$s_0 \leq s$为最小整数，使得当$\lambda = \sqrt{2 \log p/n}$时，满足$\sum_{i=1}^p \min(\beta_i^2, \lambda^2 \sigma^2) \leq s_0 \lambda^2 \sigma^2$。针对阈值Lasso的仿真结果与理论分析高度吻合。