Interactive semantic parsing based on natural language (NL) feedback, where users provide feedback to correct the parser mistakes, has emerged as a more practical scenario than the traditional one-shot semantic parsing. However, prior work has heavily relied on human-annotated feedback data to train the interactive semantic parser, which is prohibitively expensive and not scalable. In this work, we propose a new task of simulating NL feedback for interactive semantic parsing. We accompany the task with a novel feedback evaluator. The evaluator is specifically designed to assess the quality of the simulated feedback, based on which we decide the best feedback simulator from our proposed variants. On a text-to-SQL dataset, we show that our feedback simulator can generate high-quality NL feedback to boost the error correction ability of a specific parser. In low-data settings, our feedback simulator can help achieve comparable error correction performance as trained using the costly, full set of human annotations.

翻译：基于自然语言反馈的交互式语义解析（用户提供反馈以纠正解析器错误）已成为比传统单次语义解析更实用的场景。然而，先前的工作严重依赖人工标注的反馈数据来训练交互式语义解析器，这种方法成本高昂且不可扩展。本文提出了一项新任务：为交互式语义解析模拟自然语言反馈。我们为该任务配套设计了一种新颖的反馈评估器。该评估器专门用于评估模拟反馈的质量，基于此我们从提出的多种变体中确定最佳的反馈模拟器。在一个文本到SQL数据集上，我们证明反馈模拟器能够生成高质量的自然语言反馈，从而提升特定解析器的纠错能力。在低数据场景下，我们的反馈模拟器有助于实现与使用昂贵完整人工标注训练数据相当的纠错性能。