We consider the problem of parameter estimation from observations given by a generalized linear model. Spectral methods are a simple yet effective approach for estimation: they estimate the parameter via the principal eigenvector of a matrix obtained by suitably preprocessing the observations. Despite their wide use, a rigorous performance characterization of spectral estimators, as well as a principled way to preprocess the data, is available only for unstructured (i.e., i.i.d. Gaussian and Haar) designs. In contrast, real-world design matrices are highly structured and exhibit non-trivial correlations. To address this problem, we consider correlated Gaussian designs which capture the anisotropic nature of the measurements via a feature covariance matrix $\Sigma$. Our main result is a precise asymptotic characterization of the performance of spectral estimators in this setting. This then allows to identify the optimal preprocessing that minimizes the number of samples needed to meaningfully estimate the parameter. Remarkably, such an optimal spectral estimator depends on $\Sigma$ only through its normalized trace, which can be consistently estimated from the data. Numerical results demonstrate the advantage of our principled approach over previous heuristic methods. Existing analyses of spectral estimators crucially rely on the rotational invariance of the design matrix. This key assumption does not hold for correlated Gaussian designs. To circumvent this difficulty, we develop a novel strategy based on designing and analyzing an approximate message passing algorithm whose fixed point coincides with the desired spectral estimator. Our methodology is general, and opens the way to the precise characterization of spiked matrices and of the corresponding spectral methods in a variety of settings.

翻译：我们考虑由广义线性模型给出的观测数据中的参数估计问题。光谱方法是一种简单而有效的估计手段：它通过对观测数据进行适当预处理后得到的矩阵的主特征向量来估计参数。尽管光谱估计器应用广泛，但其严格的性能刻画以及数据预处理的系统化方法仅适用于非结构化（即独立同分布的高斯和哈尔）设计。然而，现实中的设计矩阵往往具有高度结构化特征并呈现非平凡的相关性。为解决这一问题，我们考虑通过特征协方差矩阵$\Sigma$来刻画测量各向异性性质的相关高斯设计。我们的主要结果是给出了此类设置下光谱估计器性能的精确渐近刻画。据此能够识别出使有意义估计参数所需样本量最小化的最优预处理方法。值得注意的是，这种最优光谱估计器仅通过其归一化迹依赖于$\Sigma$，而该归一化迹可从数据中一致估计。数值结果展示了我们系统化方法相较于以往启发式方法的优势。现有光谱估计器的分析关键依赖于设计矩阵的旋转不变性，但这一关键假设在相关高斯设计中并不成立。为克服这一困难，我们提出了一种基于设计并分析近似消息传递算法的新策略——该算法的固定点恰好对应于目标光谱估计器。我们的方法具有普适性，为精确刻画各种场景下的尖峰矩阵及其对应的光谱方法开辟了道路。