Neural reasoning accuracy improves when generating intermediate reasoning steps. However, the source of this improvement is yet unclear. Here, we investigate and factorize the benefit of generating intermediate steps for symbolic reasoning. Specifically, we decompose the reasoning strategy w.r.t. step granularity and chaining strategy. With a purely symbolic numerical reasoning dataset (e.g., A=1, B=3, C=A+3, C?), we found that the choice of reasoning strategies significantly affects the performance, with the gap becoming even larger as the extrapolation length becomes longer. Surprisingly, we also found that certain configurations lead to nearly perfect performance, even in the case of length extrapolation. Our results indicate the importance of further exploring effective strategies for neural reasoning models.

翻译：神经推理在生成中间推理步骤时，其准确率有所提升。然而，这一提升的根源尚不明确。本文针对符号推理中生成中间步骤的益处进行了探究与分解。具体而言，我们从步骤粒度和链接策略两个维度对推理策略进行分解。基于纯符号数值推理数据集（例如：A=1，B=3，C=A+3，C?），我们发现推理策略的选择显著影响模型性能，且随着外推长度的增加，性能差异进一步扩大。令人惊讶的是，我们还发现某些配置能够实现近乎完美的性能，即使在长度外推的情况下也是如此。我们的研究结果表明，进一步探索神经推理模型的有效策略具有重要意义。