Neural algorithmic reasoning is an emerging area of machine learning focusing on building models that can imitate the execution of classic algorithms, such as sorting, shortest paths, etc. One of the main challenges is to learn algorithms that are able to generalize to out-of-distribution data, in particular with significantly larger input sizes. Recent work on this problem has demonstrated the advantages of learning algorithms step-by-step, giving models access to all intermediate steps of the original algorithm. In this work, we instead focus on learning neural algorithmic reasoning only from the input-output pairs without appealing to the intermediate supervision. We propose simple but effective architectural improvements and also build a self-supervised objective that can regularise intermediate computations of the model without access to the algorithm trajectory. We demonstrate that our approach is competitive to its trajectory-supervised counterpart on tasks from the CLRS Algorithmic Reasoning Benchmark and achieves new state-of-the-art results for several problems, including sorting, where we obtain significant improvements. Thus, learning without intermediate supervision is a promising direction for further research on neural reasoners.

翻译：神经算法推理是机器学习中的一个新兴领域，专注于构建能够模仿经典算法（如排序、最短路径等）执行的模型。其主要挑战之一在于学习能够泛化到分布外数据（尤其是输入规模显著增大时）的算法。近期关于该问题的研究已证明逐步学习算法的优势，即让模型获取原始算法的所有中间步骤。在本工作中，我们转而聚焦于仅从输入-输出对中学习神经算法推理，而不依赖中间监督。我们提出了简单但有效的架构改进，并构建了一种自监督目标函数，该函数无需算法轨迹即可正则化模型的中间计算过程。实验表明，我们的方法在CLRS算法推理基准测试的任务中与轨迹监督方法具有竞争力，并在包括排序在内的多个问题上取得了新的最佳结果，其中我们获得了显著提升。因此，无需中间监督的学习是神经推理器未来研究的一个有前景方向。