It has been observed that the performances of many high-dimensional estimation problems are universal with respect to underlying sensing (or design) matrices. Specifically, matrices with markedly different constructions seem to achieve identical performance if they share the same spectral distribution and have ``generic'' singular vectors. We prove this universality phenomenon for the case of convex regularized least squares (RLS) estimators under a linear regression model with additive Gaussian noise. Our main contributions are two-fold: (1) We introduce a notion of universality classes for sensing matrices, defined through a set of deterministic conditions that fix the spectrum of the sensing matrix and precisely capture the previously heuristic notion of generic singular vectors; (2) We show that for all sensing matrices that lie in the same universality class, the dynamics of the proximal gradient descent algorithm for solving the regression problem, as well as the performance of RLS estimators themselves (under additional strong convexity conditions) are asymptotically identical. In addition to including i.i.d. Gaussian and rotational invariant matrices as special cases, our universality class also contains highly structured, strongly correlated, or even (nearly) deterministic matrices. Examples of the latter include randomly signed versions of incoherent tight frames and randomly subsampled Hadamard transforms. As a consequence of this universality principle, the asymptotic performance of regularized linear regression on many structured matrices constructed with limited randomness can be characterized by using the rotationally invariant ensemble as an equivalent yet mathematically more tractable surrogate.

翻译：许多高维估计问题的性能被发现对底层感知（或设计）矩阵具有普适性。具体而言，若矩阵共享相同的谱分布且具有“通用”奇异向量，即便构造方式迥异，其性能也似乎趋于一致。我们针对线性回归模型（含加性高斯噪声）下的凸正则化最小二乘（RLS）估计器验证了这一普适性现象。主要贡献包括两方面：（1）引入感知矩阵的普适性类概念，通过一组确定性的条件定义——这些条件固定了感知矩阵的谱分布，并精确捕捉了此前启发性描述的通用奇异向量；（2）证明对于同一普适性类中的所有感知矩阵，求解回归问题的近端梯度下降算法的动力学特性，以及RLS估计器本身的性能（在额外强凸性条件下）均具有渐近一致性。除将独立同分布高斯矩阵和旋转不变矩阵作为特例外，我们的普适性类还包含高度结构化、强相关甚至（近乎）确定性的矩阵。后者的典型例子包括非相干紧框架的随机符号化版本和随机欠采样哈达玛变换。基于这一普适性原理，可通过旋转不变系综（作为数学上更易处理的等价替代）来刻画有限随机性构造的诸多结构化矩阵上正则化线性回归的渐近性能。