Characterizing the distribution of high-dimensional statistical estimators is a challenging task, due to the breakdown of classical asymptotic theory in high dimension. This paper makes progress towards this by developing non-asymptotic distributional characterizations for approximate message passing (AMP) -- a family of iterative algorithms that prove effective as both fast estimators and powerful theoretical machinery -- for both sparse and robust regression. Prior AMP theory, which focused on high-dimensional asymptotics for the most part, failed to describe the behavior of AMP when the number of iterations exceeds $o\big({\log n}/{\log \log n}\big)$ (with $n$ the sample size). We establish the first finite-sample non-asymptotic distributional theory of AMP for both sparse and robust regression that accommodates a polynomial number of iterations. Our results derive approximate accuracy of Gaussian approximation of the AMP iterates, which improves upon all prior results and implies enhanced distributional characterizations for both optimally tuned Lasso and robust M-estimator.

翻译：在高维统计中，经典渐近理论失效，这使得刻画高维统计估计量的分布成为一项具有挑战性的任务。本文通过为近似消息传递（AMP）——一类兼具快速估计器和强大理论工具特性的迭代算法——建立非渐近分布刻画，在稀疏回归与稳健回归两方面均取得了进展。以往的AMP理论主要聚焦于高维渐近性，未能描述当迭代次数超过$o\big({\log n}/{\log \log n}\big)$（其中$n$为样本量）时AMP的行为。我们首次建立了适用于稀疏和稳健回归的AMP有限样本非渐近分布理论，该理论可容纳多项式次迭代。我们的结果推导出AMP迭代的高斯近似的近似精度，这一精度优于以往所有结果，并由此改进了最优调谐Lasso和稳健M估计量的分布刻画。