General first-order methods (GFOM) are a flexible class of iterative algorithms which update a state vector by matrix-vector multiplications and entrywise nonlinearities. A long line of work has sought to understand the large-n dynamics of GFOM, mostly focusing on "very random" input matrices and the approximate message passing (AMP) special case of GFOM whose state is asymptotically Gaussian. Yet, it has long remained unknown how to construct iterative algorithms that retain this Gaussianity for more structured inputs, or why existing AMP algorithms can be as effective for some deterministic matrices as they are for random matrices. We analyze diagrammatic expansions of GFOM via the limiting traffic distribution of the input matrix, the collection of all limiting values of permutation-invariant polynomials in the matrix entries, to obtain the following results: 1. We calculate the traffic distribution for the first non-trivial deterministic matrices, including (minor variants of) the Walsh-Hadamard and discrete sine and cosine transform matrices. This determines the limiting dynamics of GFOM on these inputs, resolving parts of longstanding conjectures of Marinari, Parisi, and Ritort (1994). 2. We design a new AMP iteration which unifies several previous AMP variants and generalizes to new input types, whose limiting dynamics are Gaussian conditional on some latent random variables. The asymptotic dynamics hold for a large and natural class of traffic distributions (encompassing both random and deterministic input matrices) and the algorithm's analysis gives a simple combinatorial interpretation of the Onsager correction, answering questions posed recently by Wang, Zhong, and Fan (2022).
翻译:通用一阶方法(General First-Order Methods, GFOM)是一类灵活的迭代算法,通过矩阵-向量乘法和逐点非线性变换更新状态向量。长期以来,大量工作试图理解GFOM在大规模系统下的动态特性,主要聚焦于“高度随机”的输入矩阵以及GFOM的特例——近似消息传递(Approximate Message Passing, AMP),其状态渐近服从高斯分布。然而,对于更具结构化的输入,如何构造保留这种高斯性的迭代算法,以及为何现有AMP算法在处理某些确定性矩阵时与随机矩阵同样有效,这些问题长期未解。我们通过输入矩阵的极限交通分布(traffic distribution)——即矩阵项中所有置换不变多项式的极限值集合——对GFOM的图展开进行解析,得到以下结果:1. 我们计算了首批非平凡确定性矩阵(包括Walsh-Hadamard变换矩阵及离散正弦、余弦变换矩阵的微小变体)的交通分布,由此确定了这些输入下GFOM的极限动态,部分解决了Marinari、Parisi和Ritort(1994)的长期猜想。2. 我们设计了一种新的AMP迭代,统一了多种先前AMP变体,并可推广至新型输入,其极限动态在给定某些潜随机变量的条件下服从高斯分布。该渐近动态适用于一大类自然的交通分布(涵盖随机与确定性输入矩阵),且算法分析为Onsager修正项提供了简单组合解释,回应了Wang、Zhong和Fan(2022)近期提出的问题。