Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computation, usually by learning to execute algorithms. A typical approach is to rely on Graph Neural Network (GNN) architectures, which encode inputs in high-dimensional latent spaces that are repeatedly transformed during the execution of the algorithm. In this work we perform a detailed analysis of the structure of the latent space induced by the GNN when executing algorithms. We identify two possible failure modes: (i) loss of resolution, making it hard to distinguish similar values; (ii) inability to deal with values outside the range observed during training. We propose to solve the first issue by relying on a softmax aggregator, and propose to decay the latent space in order to deal with out-of-range values. We show that these changes lead to improvements on the majority of algorithms in the standard CLRS-30 benchmark when using the state-of-the-art Triplet-GMPNN processor. Our code is available at https://github.com/mirjanic/nar-latent-spaces

翻译：神经算法推理（NAR）是一个研究领域，专注于设计能够可靠地捕捉经典计算的神经架构，通常通过学习执行算法来实现。典型方法依赖于图神经网络（GNN）架构，该架构将输入编码到高维潜空间中，并在算法执行过程中反复对其进行变换。在本工作中，我们对执行算法时由GNN诱导的潜空间结构进行了详细分析。我们识别出两种可能的失败模式：（i）分辨率损失，导致难以区分相似值；（ii）无法处理训练期间观察范围之外的值。我们建议通过依赖softmax聚合器来解决第一个问题，并建议通过衰减潜空间来处理超出范围的值。我们表明，当使用最先进的Triplet-GMPNN处理器时，这些变化在标准CLRS-30基准测试中的大多数算法上带来了改进。我们的代码可在 https://github.com/mirjanic/nar-latent-spaces 获取。