Sparse linear regression (SLR) is a well-studied problem in statistics where one is given a design matrix $X\in\mathbb{R}^{m\times n}$ and a response vector $y=X\theta^*+w$ for a $k$-sparse vector $\theta^*$ (that is, $\|\theta^*\|_0\leq k$) and small, arbitrary noise $w$, and the goal is to find a $k$-sparse $\widehat{\theta} \in \mathbb{R}^n$ that minimizes the mean squared prediction error $\frac{1}{m}\|X\widehat{\theta}-X\theta^*\|^2_2$. While $\ell_1$-relaxation methods such as basis pursuit, Lasso, and the Dantzig selector solve SLR when the design matrix is well-conditioned, no general algorithm is known, nor is there any formal evidence of hardness in an average-case setting with respect to all efficient algorithms. We give evidence of average-case hardness of SLR w.r.t. all efficient algorithms assuming the worst-case hardness of lattice problems. Specifically, we give an instance-by-instance reduction from a variant of the bounded distance decoding (BDD) problem on lattices to SLR, where the condition number of the lattice basis that defines the BDD instance is directly related to the restricted eigenvalue condition of the design matrix, which characterizes some of the classical statistical-computational gaps for sparse linear regression. Also, by appealing to worst-case to average-case reductions from the world of lattices, this shows hardness for a distribution of SLR instances; while the design matrices are ill-conditioned, the resulting SLR instances are in the identifiable regime. Furthermore, for well-conditioned (essentially) isotropic Gaussian design matrices, where Lasso is known to behave well in the identifiable regime, we show hardness of outputting any good solution in the unidentifiable regime where there are many solutions, assuming the worst-case hardness of standard and well-studied lattice problems.

翻译：稀疏线性回归（SLR）是统计学中一个被深入研究的课题：给定设计矩阵 $X\in\mathbb{R}^{m\times n}$ 和响应向量 $y=X\theta^*+w$，其中 $\theta^*$ 是一个 $k$-稀疏向量（即 $\|\theta^*\|_0\leq k$），$w$ 为微小任意噪声，目标是找到一个 $k$-稀疏的 $\widehat{\theta} \in \mathbb{R}^n$，以最小化均方预测误差 $\frac{1}{m}\|X\widehat{\theta}-X\theta^*\|^2_2$。当设计矩阵良态时，诸如基追踪、Lasso 和 Dantzig 选择器等 $\ell_1$ 松弛方法可以求解 SLR；然而，对于所有高效算法，目前尚无通用算法被提出，也缺乏在平均情形下关于其计算难度的形式化证据。我们基于格问题在最坏情形下的困难性，为 SLR 相对于所有高效算法的平均情形难度提供了证据。具体而言，我们给出了从格上有界距离解码（BDD）问题的一个变体到 SLR 的实例间归约，其中定义 BDD 实例的格基的条件数直接关系到设计矩阵的限制特征值条件——该条件刻画了稀疏线性回归中一些经典的统计-计算间隙。此外，通过借助格领域中最坏情形到平均情形的归约，这证明了对于一类 SLR 实例分布的困难性；尽管设计矩阵是病态的，但所得的 SLR 实例仍处于可识别区域。进一步地，对于良态（基本）各向同性的高斯设计矩阵（已知 Lasso 在可识别区域表现良好），我们证明了在存在多个解的非可识别区域中输出任何良好解的困难性，其前提是标准且被深入研究的格问题在最坏情形下是困难的。