In-context learning has shown great success in i.i.d semantic parsing splits, where the training and test sets are drawn from the same distribution. In this setup, models are typically prompted with demonstrations that are similar to the input utterance. However, in the setup of compositional generalization, where models are tested on outputs with structures that are absent from the training set, selecting similar demonstrations is insufficient, as often no example will be similar enough to the input. In this work, we propose a method to select diverse demonstrations that aims to collectively cover all of the structures required in the output program, in order to encourage the model to generalize to new structures from these demonstrations. We empirically show that combining diverse demonstrations with in-context learning substantially improves performance across three compositional generalization semantic parsing datasets in the pure in-context learning setup and when combined with finetuning.

翻译：上下文学习在独立同分布的语义解析分割中取得了巨大成功，其中训练集和测试集来自同一分布。在此设定下，模型通常通过选择与输入话语相似的示例进行提示。然而，在组合泛化设定中，模型需对训练集中未出现结构的输出进行测试，此时选择相似示例效果不足，因为往往不存在与输入足够相似的示例。本研究提出一种选择多样化示例的方法，旨在共同覆盖输出程序所需的所有结构，从而促使模型从这些示例泛化至新结构。实验表明，在纯上下文学习设定及结合微调时，将多样化示例与上下文学习相结合可显著提升三个组合泛化语义解析数据集的性能。