We study high-dimensional least-squares regression within a subgaussian statistical learning framework with heterogeneous noise. It includes $s$-sparse and $r$-low-rank least-squares regression when a fraction $\epsilon$ of the labels are adversarially contaminated. We also present a novel theory of trace-regression with matrix decomposition based on a new application of the product process. For these problems, we show novel near-optimal "subgaussian" estimation rates of the form $r(n,d_{e})+\sqrt{\log(1/\delta)/n}+\epsilon\log(1/\epsilon)$, valid with probability at least $1-\delta$. Here, $r(n,d_{e})$ is the optimal uncontaminated rate as a function of the effective dimension $d_{e}$ but independent of the failure probability $\delta$. These rates are valid uniformly on $\delta$, i.e., the estimators' tuning do not depend on $\delta$. Lastly, we consider noisy robust matrix completion with non-uniform sampling. If only the low-rank matrix is of interest, we present a novel near-optimal rate that is independent of the corruption level $a$. Our estimators are tractable and based on a new "sorted" Huber-type loss. No information on $(s,r,\epsilon,a)$ are needed to tune these estimators. Our analysis makes use of novel $\delta$-optimal concentration inequalities for the multiplier and product processes which could be useful elsewhere. For instance, they imply novel sharp oracle inequalities for Lasso and Slope with optimal dependence on $\delta$. Numerical simulations confirm our theoretical predictions. In particular, "sorted" Huber regression can outperform classical Huber regression.

翻译：我们研究异质噪声下亚高斯统计学习框架中的高维最小二乘回归问题。该框架涵盖当标签被比例为$\epsilon$的对抗性污染时，基于$s$-稀疏和$r$-低秩约束的最小二乘回归。同时，我们通过乘积过程的新应用，提出了基于矩阵分解的迹回归新理论。针对这些问题，我们证明了具有形式$r(n,d_{e})+\sqrt{\log(1/\delta)/n}+\epsilon\log(1/\epsilon)$的新型近最优"亚高斯"估计率，该估计率以至少$1-\delta$的概率成立。其中$r(n,d_{e})$是有效维度$d_{e}$的函数（与失败概率$\delta$无关）所对应的最优无污染估计率。这些估计率对$\delta$一致成立，即估计器的调参不依赖于$\delta$。最后，我们研究非均匀采样下的含噪稳健矩阵补全问题。若仅关注低秩矩阵，我们提出与污染水平$a$无关的新型近最优估计率。我们的估计器具有可解性，并基于新型"排序"Huber型损失函数。这些估计器的调参无需关于$(s,r,\epsilon,a)$的先验信息。通过引入乘性过程和乘积过程的$\delta$-最优浓度不等式（该不等式可能在其他领域具有应用价值），我们完成了理论分析。例如，这些不等式可推导出Lasso和Slope方法中关于$\delta$最优依赖的锐化Oracle不等式。数值实验验证了理论预测，特别地，"排序"Huber回归能显著优于传统Huber回归。