In high-dimensional regression, we attempt to estimate a parameter vector $\beta_0\in\mathbb{R}^p$ from $n\lesssim p$ observations $\{(y_i,x_i)\}_{i\leq n}$ where $x_i\in\mathbb{R}^p$ is a vector of predictors and $y_i$ is a response variable. A well-established approach uses convex regularizers to promote specific structures (e.g. sparsity) of the estimate $\widehat{\beta}$, while allowing for practical algorithms. Theoretical analysis implies that convex penalization schemes have nearly optimal estimation properties in certain settings. However, in general the gaps between statistically optimal estimation (with unbounded computational resources) and convex methods are poorly understood. We show that when the statistican has very simple structural information about the distribution of the entries of $\beta_0$, a large gap frequently exists between the best performance achieved by any convex regularizer satisfying a mild technical condition and either (i) the optimal statistical error or (ii) the statistical error achieved by optimal approximate message passing algorithms. Remarkably, a gap occurs at high enough signal-to-noise ratio if and only if the distribution of the coordinates of $\beta_0$ is not log-concave. These conclusions follow from an analysis of standard Gaussian designs. Our lower bounds are expected to be generally tight, and we prove tightness under certain conditions.

翻译：在高维回归中,我们试图从 $(y_i,x_i)\\i\i\leq n}$(x_i\in\mathb{R ⁇ p$) 是预测器的矢量, $(i_i)是响应变量。 一种成熟的方法使用 comvex 正规化器促进特定结构( 例如, 缩放) $( 全域) 的估算值, 同时允许实际的算法。 理论分析意味着 comvex 惩罚性计划在某些环境下几乎具有最佳的估计属性 $( y_i, x_i)\\ i\ i\ leq n}$( 美元 美元 美元 ) 。 然而, 总的来说, 统计性的最佳估计( 有未限制的计算资源) 和 convex 方法之间的缺口很少被理解。 当统计性结构信息非常简单的关于 $( beta_ 0 美元) 条目的分布结构信息时, 经常存在一个巨大的差距: 任何符合较轻技术条件的 convex manx manferfor manizerizer 和 (i) lax lax lax lax lax lax lax lax lax lax lax lax) 通常不是 最接近的精确的统计性分析。