We investigate high-dimensional sparse regression when both the noise and the design matrix exhibit heavy-tailed behavior. Standard algorithms typically fail in this regime, as heavy-tailed covariates distort the empirical risk geometry. We propose a unified framework, Robust Iterative Gradient descent with Hard Thresholding (RIGHT), which employs a robust gradient estimator to bypass the need for higher-order moment conditions. Our analysis reveals a fundamental decoupling phenomenon: in linear regression, the estimation error rate is governed by the noise tail index, while the sample complexity required for stability is governed by the design tail index. This implies that while heavy-tailed noise limits precision, heavy-tailed designs primarily raise the sample size barrier for convergence. In contrast, for logistic regression, we show that the bounded gradient naturally robustifies the estimator against heavy-tailed designs, restoring standard parametric rates. We derive matching minimax lower bounds to prove that RIGHT achieves optimal estimation accuracy and sample complexity across these regimes, without requiring sample splitting or the existence of the population risk.

翻译：本文研究高维稀疏回归问题，其中噪声和设计矩阵均呈现重尾特性。在此情形下，标准算法通常失效，因为重尾协变量会扭曲经验风险的几何结构。我们提出一个统一框架——鲁棒迭代梯度下降硬阈值算法（RIGHT），该算法采用鲁棒梯度估计器，从而无需高阶矩条件。我们的分析揭示了一个基本解耦现象：在线性回归中，估计误差率由噪声的尾指数决定，而稳定性所需的样本复杂度则由设计的尾指数决定。这意味着，重尾噪声会限制估计精度，而重尾设计主要提高收敛所需的样本量门槛。相比之下，对于逻辑回归，我们证明有界梯度自然使估计器对重尾设计具有鲁棒性，从而恢复标准的参数速率。我们推导了匹配的极小极大下界，证明RIGHT在这些情形下无需样本分割或总体风险存在即可达到最优的估计精度和样本复杂度。