Despite the success of neural models in solving reasoning tasks, their compositional generalization capabilities remain unclear. In this work, we propose a new setting of the structured explanation generation task to facilitate compositional reasoning research. Previous works found that symbolic methods achieve superior compositionality by using pre-defined inference rules for iterative reasoning. But these approaches rely on brittle symbolic transfers and are restricted to well-defined tasks. Hence, we propose a dynamic modularized reasoning model, MORSE, to improve the compositional generalization of neural models. MORSE factorizes the inference process into a combination of modules, where each module represents a functional unit. Specifically, we adopt modularized self-attention to dynamically select and route inputs to dedicated heads, which specializes them to specific functions. We conduct experiments for increasing lengths and shapes of reasoning trees on two benchmarks to test MORSE's compositional generalization abilities, and find it outperforms competitive baselines. Model ablation and deeper analyses show the effectiveness of dynamic reasoning modules and their generalization abilities.

翻译：尽管神经模型在解决推理任务方面取得了成功，但其组合泛化能力仍不明确。在这项工作中，我们提出了结构化解释生成任务的新设定，以促进组合推理研究。先前的研究发现，符号方法通过使用预定义的推理规则进行迭代推理，实现了优越的组合性。但这些方法依赖于脆弱的符号转换，且局限于定义明确的任务。因此，我们提出了一种动态模块化推理模型MORSE，以提升神经模型的组合泛化能力。MORSE将推理过程分解为多个模块的组合，每个模块代表一个功能单元。具体而言，我们采用模块化自注意力机制，动态选择输入并将其路由到专用注意力头，从而使这些注意力头专门执行特定功能。我们在两个基准数据集上，针对推理树长度和形状不断增长的情况进行了实验，以测试MORSE的组合泛化能力，并发现其优于竞争基线方法。模型消融实验和深入分析显示了动态推理模块的有效性及其泛化能力。