Solving a linear system $Ax=b$ is a fundamental scientific computing primitive for which numerous solvers and preconditioners have been developed. These come with parameters whose optimal values depend on the system being solved and are often impossible or too expensive to identify; thus in practice sub-optimal heuristics are used. We consider the common setting in which many related linear systems need to be solved, e.g. during a single numerical simulation. In this scenario, can we sequentially choose parameters that attain a near-optimal overall number of iterations, without extra matrix computations? We answer in the affirmative for Successive Over-Relaxation (SOR), a standard solver whose parameter $\omega$ has a strong impact on its runtime. For this method, we prove that a bandit online learning algorithm--using only the number of iterations as feedback--can select parameters for a sequence of instances such that the overall cost approaches that of the best fixed $\omega$ as the sequence length increases. Furthermore, when given additional structural information, we show that a contextual bandit method asymptotically achieves the performance of the instance-optimal policy, which selects the best $\omega$ for each instance. Our work provides the first learning-theoretic treatment of high-precision linear system solvers and the first end-to-end guarantees for data-driven scientific computing, demonstrating theoretically the potential to speed up numerical methods using well-understood learning algorithms.

翻译：求解线性系统 $Ax=b$ 是基础的科学计算原语，为此已开发出众多求解器和预处理器。这些工具带有参数，其最优值取决于待求解的系统，且通常无法或成本过高而难以确定；因此，实践中常采用次优启发式方法。我们考虑一种常见场景：需要求解许多相关线性系统（例如在单次数值模拟过程中）。在此情境下，我们能否在不额外进行矩阵计算的情况下，顺序选择参数以实现接近最优的整体迭代次数？我们针对逐次超松弛法（SOR）给出肯定回答——该标准求解器的参数 $\omega$ 对其运行时间有显著影响。针对该方法，我们证明：一种仅以迭代次数作为反馈的在线学习算法（bandit）能够为实例序列选择参数，使得随着序列长度增加，整体代价趋近于最佳固定 $\omega$ 的代价。此外，当提供额外结构信息时，我们证明一种上下文bandit方法能渐近达到实例最优策略的性能——即为每个实例选取最佳 $\omega$。本文首次为高精度线性系统求解器提供学习理论框架，并为数据驱动科学计算提供首项端到端保证，从理论上展示了利用成熟学习算法加速数值方法的潜力。