In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a line of work often referred to as neural algorithmic reasoning. The goal is to integrate algorithmic reasoning capabilities into larger neural pipelines. Many such architectures are based on (message-passing) graph neural networks (MPNNs), owing to their permutation equivariance and ability to deal with sparsity and variable-sized inputs. However, existing work is either largely empirical and lacks formal guarantees or it focuses solely on expressivity, leaving open the question of when and how such architectures generalize beyond a finite training set. In this work, we propose a general theoretical framework that characterizes the sufficient conditions under which MPNNs can learn an algorithm from a training set of small instances and provably approximate its behavior on inputs of arbitrary size. Our framework applies to a broad class of algorithms, including single-source shortest paths, minimum spanning trees, and general dynamic programming problems, such as the $0$-$1$ knapsack problem. In addition, we establish impossibility results for a wide range of algorithmic tasks, showing that standard MPNNs cannot learn them, and we derive more expressive MPNN-like architectures that overcome these limitations. Finally, we refine our analysis for the Bellman-Ford algorithm, yielding a substantially smaller required training set and significantly extending the recent work of Nerem et al. [2025] by allowing for a differentiable regularization loss. Empirical results largely support our theoretical findings.

翻译：近年来，理解神经网络架构学习执行离散算法的能力日益受到关注，这一研究方向常被称为神经算法推理。其目标是将算法推理能力整合到更大的神经处理流程中。许多此类架构基于（消息传递）图神经网络（MPNNs），因为它们具有置换等变性，并能处理稀疏性和可变尺寸的输入。然而，现有研究要么主要是经验性的且缺乏形式化保证，要么仅关注表达能力，未能解答此类架构何时以及如何泛化到有限训练集之外的问题。在本研究中，我们提出了一个通用的理论框架，该框架刻画了MPNNs能够从包含小实例的训练集中学习算法，并能在任意尺寸的输入上可证明地近似其行为的充分条件。我们的框架适用于广泛的算法类别，包括单源最短路径、最小生成树以及一般的动态规划问题，例如$0$-$1$背包问题。此外，我们为广泛的算法任务建立了不可能性结果，表明标准的MPNNs无法学习这些任务，并推导出更具表达能力的类MPNN架构以克服这些限制。最后，我们针对Bellman-Ford算法细化了分析，得到了所需训练集规模的大幅缩减，并通过引入可微的正则化损失，显著扩展了Nerem等人[2025]近期的工作。实证结果在很大程度上支持了我们的理论发现。