Despite remarkable progress in text-to-SQL semantic parsing in recent years, the performance of existing parsers is still far from perfect. Specifically, modern text-to-SQL parsers based on deep learning are often over-confident, thus casting doubt on their trustworthiness when deployed for real use. In this paper, we propose a parser-independent error detection model for text-to-SQL semantic parsing. Using a language model of code as its bedrock, we enhance our error detection model with graph neural networks that learn structural features of both natural language questions and SQL queries. We train our model on realistic parsing errors collected from a cross-domain setting, which leads to stronger generalization ability. Experiments with three strong text-to-SQL parsers featuring different decoding mechanisms show that our approach outperforms parser-dependent uncertainty metrics. Our model could also effectively improve the performance and usability of text-to-SQL semantic parsers regardless of their architectures. (Our implementation is available at https://github.com/OSU-NLP-Group/Text2SQL-Error-Detection)

翻译：尽管近年来文本到SQL语义解析取得了显著进展，现有解析器的性能仍远非完美。具体而言，基于深度学习的现代文本到SQL解析器往往表现出过度自信，这在实际部署中对其可信度提出了质疑。本文提出了一种与解析器无关的文本到SQL语义解析错误检测模型。该模型以代码语言模型为基础，通过图神经网络学习自然语言问题和SQL查询的结构特征，从而增强错误检测能力。我们在跨领域场景下收集的实际解析错误上训练模型，这赋予了模型更强的泛化能力。针对三种采用不同解码机制的强基线文本到SQL解析器进行的实验表明，我们的方法优于与解析器相关的不确定性指标。此外，无论解析器的架构如何，我们的模型均能有效提升文本到SQL语义解析器的性能与可用性。（我们的实现代码详见：https://github.com/OSU-NLP-Group/Text2SQL-Error-Detection）