Zero-shot NL2SQL is crucial in achieving natural language to SQL that is adaptive to new environments (e.g., new databases, new linguistic phenomena or SQL structures) with zero annotated NL2SQL samples from such environments. Existing approaches either fine-tune pre-trained language models (PLMs) based on annotated data or use prompts to guide fixed large language models (LLMs) such as ChatGPT. PLMs can perform well in schema alignment but struggle to achieve complex reasoning, while LLMs is superior in complex reasoning tasks but cannot achieve precise schema alignment. In this paper, we propose a ZeroNL2SQL framework that combines the complementary advantages of PLMs and LLMs for supporting zero-shot NL2SQL. ZeroNL2SQL first uses PLMs to generate an SQL sketch via schema alignment, then uses LLMs to fill the missing information via complex reasoning. Moreover, in order to better align the generated SQL queries with values in the given database instances, we design a predicate calibration method to guide the LLM in completing the SQL sketches based on the database instances and select the optimal SQL query via an execution-based strategy. Comprehensive experiments show that ZeroNL2SQL can achieve the best zero-shot NL2SQL performance on real-world benchmarks. Specifically, ZeroNL2SQL outperforms the state-of-the-art PLM-based methods by 3.2% to 13% and exceeds LLM-based methods by 10% to 20% on execution accuracy.
翻译:零样本NL2SQL对于实现自然语言到SQL查询的转换至关重要,它能够适应新环境(例如新数据库、新语言现象或SQL结构),且无需来自该环境的标注NL2SQL样本。现有方法要么基于标注数据微调预训练语言模型(PLMs),要么使用提示词引导固定的大型语言模型(LLMs),如ChatGPT。PLMs在模式对齐方面表现良好,但难以实现复杂推理;而LLMs在复杂推理任务中具有优势,但无法实现精确的模式对齐。本文提出了一种ZeroNL2SQL框架,结合PLMs与LLMs的互补优势,用于支持零样本NL2SQL。ZeroNL2SQL首先通过模式对齐使用PLMs生成SQL草图,然后利用LLMs通过复杂推理填充缺失信息。此外,为了更好地将生成的SQL查询与给定数据库实例中的值对齐,我们设计了一种谓词校准方法,指导LLM基于数据库实例完成SQL草图,并通过基于执行的策略选择最优SQL查询。全面实验表明,ZeroNL2SQL能够在真实世界基准上实现最佳的零样本NL2SQL性能。具体而言,ZeroNL2SQL在执行准确率上比最先进的基于PLM的方法高出3.2%至13%,比基于LLM的方法高出10%至20%。