Seq2seq models have been shown to struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. Taking inspiration from grammar-based models that excel at compositional generalisation, we present a flexible end-to-end differentiable neural model that composes two structural operations: a fertility step, which we introduce in this work, and a reordering step based on previous work (Wang et al., 2021). To ensure differentiability, we use the expected value of each step. Our model outperforms seq2seq models by a wide margin on challenging compositional splits of realistic semantic parsing tasks that require generalisation to longer examples. It also compares favourably to other models targeting compositional generalisation.

翻译：序列到序列模型在组合泛化方面表现不佳，即难以泛化到训练中未见的新结构或更复杂结构。受擅长组合泛化的基于语法模型的启发，我们提出了一种灵活的端到端可微神经模型，该模型结合了两种结构操作：本文引入的语篇生成步骤，以及基于先前工作（Wang等，2021）的重排序步骤。为确保可微性，我们使用每个步骤的期望值。在需要泛化到更长示例的现实语义解析任务的具有挑战性的组合拆分上，我们的模型大幅优于序列到序列模型，并且与其他针对组合泛化的模型相比也具有优势。