Natural Language Interfaces for Databases empower non-technical users to interact with data using natural language (NL). Advanced approaches, utilizing either neural sequence-to-sequence or more recent sophisticated large-scale language models, typically implement NL to SQL (NL2SQL) translation in an end-to-end fashion. However, like humans, these end-to-end translation models may not always generate the best SQL output on their first try. In this paper, we propose CycleSQL, an iterative framework designed for end-to-end translation models to autonomously generate the best output through self-evaluation. The main idea of CycleSQL is to introduce data-grounded NL explanations of query results as self-provided feedback, and use the feedback to validate the correctness of the translation iteratively, hence improving the overall translation accuracy. Extensive experiments, including quantitative and qualitative evaluations, are conducted to study CycleSQL by applying it to seven existing translation models on five widely used benchmarks. The results show that 1) the feedback loop introduced in CycleSQL can consistently improve the performance of existing models, and in particular, by applying CycleSQL to RESDSQL, obtains a translation accuracy of 82.0% (+2.6%) on the validation set, and 81.6% (+3.2%) on the test set of Spider benchmark; 2) the generated NL explanations can also provide insightful information for users, aiding in the comprehension of translation results and consequently enhancing the interpretability of NL2SQL translation.
翻译:数据库的自然语言接口使非技术用户能够使用自然语言(NL)与数据进行交互。先进的方法,无论是利用神经序列到序列模型还是更近期复杂的大规模语言模型,通常以端到端方式实现自然语言到SQL(NL2SQL)的翻译。然而,与人类类似,这些端到端翻译模型可能并不总是能在首次尝试时生成最佳的SQL输出。在本文中,我们提出了CycleSQL,一个为端到端翻译模型设计的迭代框架,旨在通过自我评估自主生成最佳输出。CycleSQL的核心思想是引入基于数据的查询结果自然语言解释作为自我提供的反馈,并利用该反馈迭代验证翻译的正确性,从而提高整体翻译准确率。我们进行了广泛的实验,包括定量和定性评估,通过将CycleSQL应用于五个广泛使用的基准测试上的七个现有翻译模型来研究其性能。结果表明:1)CycleSQL中引入的反馈循环能够持续提升现有模型的性能,特别是将CycleSQL应用于RESDSQL时,在Spider基准测试的验证集上获得了82.0%(+2.6%)的翻译准确率,在测试集上获得了81.6%(+3.2%)的翻译准确率;2)生成的自然语言解释还能为用户提供有洞察力的信息,有助于理解翻译结果,从而增强NL2SQL翻译的可解释性。