We study the performance of empirical risk minimization on the $p$-norm linear regression problem for $p \in (1, \infty)$. We show that, in the realizable case, under no moment assumptions, and up to a distribution-dependent constant, $O(d)$ samples are enough to exactly recover the target. Otherwise, for $p \in [2, \infty)$, and under weak moment assumptions on the target and the covariates, we prove a high probability excess risk bound on the empirical risk minimizer whose leading term matches, up to a constant that depends only on $p$, the asymptotically exact rate. We extend this result to the case $p \in (1, 2)$ under mild assumptions that guarantee the existence of the Hessian of the risk at its minimizer.

翻译：我们研究了在$p \in (1, \infty)$下的$p$范数线性回归问题中经验风险最小化的性能。我们证明，在可实现情况下，无需矩假设且与分布相关的常数项成立时，仅需$O(d)$个样本即可精确恢复目标值。此外，对于$p \in [2, \infty)$，在目标变量与协变量满足弱矩假设的条件下，我们证明了经验风险最小化器的高概率超额风险界，其主导项与渐近精确速率匹配，仅相差一个仅依赖于$p$的常数。我们将这一结果推广至$p \in (1, 2)$的情形，该推广基于保证风险最小化点处海森矩阵存在的温和假设。