Seq2seq models have been shown to struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions of phenomena that the model handles correctly in isolation. We phrase semantic parsing as a two-step process: we first tag each input token with a multiset of output tokens. Then we arrange the tokens into an output sequence using a new way of parameterizing and predicting permutations. We formulate predicting a permutation as solving a regularized linear program and we backpropagate through the solver. In contrast to prior work, our approach does not place a priori restrictions on possible permutations, making it very expressive. Our model outperforms pretrained seq2seq models and prior work on realistic semantic parsing tasks that require generalization to longer examples. We also outperform non-tree-based models on structural generalization on the COGS benchmark. For the first time, we show that a model without an inductive bias provided by trees achieves high accuracy on generalization to deeper recursion.

翻译：序列到序列模型在语义解析中已被证明难以实现组合泛化，即泛化到模型单独正确处理现象的未见组合。我们将语义解析表述为两步过程：首先，我们用输出标记的多集标注每个输入标记；然后，通过一种新的参数化与预测排列方式，将标记排列成输出序列。我们将排列预测表述为求解正则化线性规划问题，并通过求解器进行反向传播。与先前工作不同，我们的方法不对可能排列施加先验限制，因而具有极高的表达力。我们的模型在需要泛化到更长实例的现实语义解析任务上，优于预训练序列到序列模型及先前工作。在COGS基准的结构泛化任务上，我们亦优于非树基模型。首次，我们展示了无树归纳偏好的模型能在更深递归泛化上取得高精度。