A generic out-of-sample error estimate is proposed for robust $M$-estimators regularized with a convex penalty in high-dimensional linear regression where $(X,y)$ is observed and $p,n$ are of the same order. If $\psi$ is the derivative of the robust data-fitting loss $\rho$, the estimate depends on the observed data only through the quantities $\hat\psi = \psi(y-X\hat\beta)$, $X^\top \hat\psi$ and the derivatives $(\partial/\partial y) \hat\psi$ and $(\partial/\partial y) X\hat\beta$ for fixed $X$. The out-of-sample error estimate enjoys a relative error of order $n^{-1/2}$ in a linear model with Gaussian covariates and independent noise, either non-asymptotically when $p/n\le \gamma$ or asymptotically in the high-dimensional asymptotic regime $p/n\to\gamma'\in(0,\infty)$. General differentiable loss functions $\rho$ are allowed provided that $\psi=\rho'$ is 1-Lipschitz. The validity of the out-of-sample error estimate holds either under a strong convexity assumption, or for the $\ell_1$-penalized Huber M-estimator if the number of corrupted observations and sparsity of the true $\beta$ are bounded from above by $s_*n$ for some small enough constant $s_*\in(0,1)$ independent of $n,p$. For the square loss and in the absence of corruption in the response, the results additionally yield $n^{-1/2}$-consistent estimates of the noise variance and of the generalization error. This generalizes, to arbitrary convex penalty, estimates that were previously known for the Lasso.

翻译：针对高维线性回归中带凸罚项正则化的鲁棒$M$估计，本文提出了一种通用的样本外误差估计方法，其中观测到$(X,y)$且$p,n$同阶。若$\psi$为稳健数据拟合损失$\rho$的导数，该估计仅通过量$\hat\psi = \psi(y-X\hat\beta)$、$X^\top \hat\psi$以及固定$X$下的导数$(\partial/\partial y) \hat\psi$和$(\partial/\partial y) X\hat\beta$依赖于观测数据。在具有高斯协变量和独立噪声的线性模型中，该样本外误差估计在$p/n\le \gamma$的非渐近情形或高维渐近机制$p/n\to\gamma'\in(0,\infty)$下享有$n^{-1/2}$阶的相对误差。允许使用一般的可微损失函数$\rho$，前提是$\psi=\rho'$为1-Lipschitz函数。该样本外误差估计的有效性在强凸假设下成立，或对于$\ell_1$罚Huber M估计，当受污染观测值和真实$\beta$的稀疏性均被$s_*n$（其中$s_*\in(0,1)$为独立于$n,p$的足够小常数）上界约束时成立。对于平方损失且响应无污染的情形，所得结果额外给出了噪声方差和泛化误差的$n^{-1/2}$相合估计。这推广了先前仅适用于Lasso的估计方法至任意凸罚项。