Solving a linear system $Ax=b$ is a fundamental scientific computing primitive for which numerous solvers and preconditioners have been developed. These come with parameters whose optimal values depend on the system being solved and are often impossible or too expensive to identify; thus in practice sub-optimal heuristics are used. We consider the common setting in which many related linear systems need to be solved, e.g. during a single numerical simulation. In this scenario, can we sequentially choose parameters that attain a near-optimal overall number of iterations, without extra matrix computations? We answer in the affirmative for Successive Over-Relaxation (SOR), a standard solver whose parameter $\omega$ has a strong impact on its runtime. For this method, we prove that a bandit online learning algorithm -- using only the number of iterations as feedback -- can select parameters for a sequence of instances such that the overall cost approaches that of the best fixed $\omega$ as the sequence length increases. Furthermore, when given additional structural information, we show that a contextual bandit method asymptotically achieves the performance of the instance-optimal policy, which selects the best $\omega$ for each instance. Our work provides the first learning-theoretic treatment of high-precision linear system solvers and the first end-to-end guarantees for data-driven scientific computing, demonstrating theoretically the potential to speed up numerical methods using well-understood learning algorithms.

翻译：求解线性方程组 $Ax=b$ 是科学计算中的基础原语，为此已开发出众多求解器和预处理器。这些方法带有参数，其最优值取决于所求解的系统，且通常难以或代价过高地确定，因此实践中常采用次优的启发式方法。我们考虑一个常见场景：需要求解许多相关的线性系统，例如在单次数值模拟过程中。在此情形下，我们能否在不进行额外矩阵计算的前提下，顺序选择参数以实现接近最优的总迭代次数？我们针对连续超松弛法（SOR）给出了肯定回答——这是一种标准求解器，其参数 $\omega$ 对运行时间有显著影响。对于该方法，我们证明：一种仅以迭代次数作为反馈的赌博机在线学习算法，能够为实例序列选择参数，使得总代价随序列长度增加而逼近最优固定 $\omega$ 对应的代价。此外，当提供额外结构信息时，我们证明上下文赌博机方法能在渐近意义上达到实例最优策略的性能，即为每个实例选择最佳 $\omega$。我们的工作首次为高精度线性系统求解器提供了学习理论视角，并首次建立了数据驱动科学计算的端到端保证，从理论上展示了利用成熟学习算法加速数值方法的潜力。