Interactive semantic parsing based on natural language (NL) feedback, where users provide feedback to correct the parser mistakes, has emerged as a more practical scenario than the traditional one-shot semantic parsing. However, prior work has heavily relied on human-annotated feedback data to train the interactive semantic parser, which is prohibitively expensive and not scalable. In this work, we propose a new task of simulating NL feedback for interactive semantic parsing. We accompany the task with a novel feedback evaluator. The evaluator is specifically designed to assess the quality of the simulated feedback, based on which we decide the best feedback simulator from our proposed variants. On a text-to-SQL dataset, we show that our feedback simulator can generate high-quality NL feedback to boost the error correction ability of a specific parser. In low-data settings, our feedback simulator can help achieve comparable error correction performance as trained using the costly, full set of human annotations.

翻译：基于自然语言反馈的交互式语义解析（用户通过提供反馈来纠正解析器错误）已成为比传统一次性语义解析更实用的场景。然而，先前的研究严重依赖人工标注的反馈数据来训练交互式语义解析器，这成本高昂且难以扩展。本文提出一项新任务：模拟面向交互式语义解析的自然语言反馈。我们为该任务设计了一种新型反馈评估器。该评估器专门用于评估模拟反馈的质量，据此我们从提出的多种变体中选出最佳反馈模拟器。在文本到SQL数据集上的实验表明，我们的反馈模拟器能生成高质量的自然语言反馈，从而提升特定解析器的纠错能力。在低数据场景下，使用我们的反馈模拟器可达到与使用昂贵完整人工标注数据训练时相当的纠错性能。