Anderson acceleration (AA) is a technique for accelerating the convergence of an underlying fixed-point iteration. AA is widely used within computational science, with applications ranging from electronic structure calculation to the training of neural networks. Despite AA's widespread use, relatively little is understood about it theoretically. An important and unanswered question in this context is: To what extent can AA actually accelerate convergence of the underlying fixed-point iteration? While simple enough to state, this question appears rather difficult to answer. For example, it is unanswered even in the simplest (non-trivial) case where the underlying fixed-point iteration consists of applying a two-dimensional affine function. In this note we consider a restarted variant of AA applied to solve symmetric linear systems with restart window of size one. Several results are derived from the analytical solution of a nonlinear eigenvalue problem characterizing residual propagation of the AA iteration. This includes a complete characterization of the method to solve $2 \times 2$ linear systems, rigorously quantifying how the asymptotic convergence factor depends on the initial iterate, and quantifying by how much AA accelerates the underlying fixed-point iteration. We also prove that even if the underlying fixed-point iteration diverges, the associated AA iteration may still converge.

翻译：安德森加速（AA）是一种用于加速基础不动点迭代收敛性的技术。该方法广泛应用于计算科学领域，其应用范围涵盖电子结构计算到神经网络训练。尽管AA得到普遍应用，但其理论理解仍相对有限。一个尚未解决的重要问题是：AA在多大程度上能够真正加速基础不动点迭代的收敛性？虽然问题表述简单，但实际解答颇具难度。例如，即使是最简单（非平凡）的二维仿射函数迭代情形，该问题仍未得到解答。本文研究窗口大小为1的重启型AA方法求解对称线性系统的收敛特性。通过解析求解表征AA迭代残差传播的非线性特征值问题，我们推导出多项关键结论：完整刻画了该方法求解$2 \times 2$线性系统的收敛行为；严格量化了渐近收敛因子与初始迭代值之间的依赖关系；定量评估了AA对基础不动点迭代的加速效果。此外，我们证明即使基础不动点迭代发散，相应的AA迭代仍可能保持收敛。