One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases. For the task of converting natural language text to SQL queries, Text-to-SQL, adaptation of LLMs is of paramount importance, both in in-context learning and fine-tuning settings, depending on the amount of adaptation data used. In this paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is based on an execution-based self-consistency prompting approach designed for Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our best knowledge is the first to outperform previous state-of-the-art with fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying SQL-PaLM to real-world scenarios we further evaluate its robustness on other challenging variants of Spider and demonstrate the superior generalization capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate the impressive intelligent capabilities and various success enablers of LLM-based Text-to-SQL.

翻译：大语言模型（LLMs）的一项令人瞩目的涌现能力是生成代码，包括数据库的结构化查询语言（SQL）。对于将自然语言文本转换为SQL查询的任务（即文本到SQL），LLMs的适配至关重要——无论是在情境学习还是微调设置中，其适配效果均取决于所使用的适配数据量。本文提出一种基于PaLM-2的LLM文本到SQL模型SQL-PaLM，在两种设置下均推动了当前最优水平的发展。少样本SQL-PaLM采用一种专为文本到SQL设计的基于执行一致性的提示方法，在Spider数据集上实现了77.3%的测试套件准确率——据我们所知，这是首次以4%的显著差距超越先前经过微调的最优模型。此外，我们证明经过微调的SQL-PaLM进一步提升了1%的准确率。为将SQL-PaLM应用于真实场景，我们进一步评估了其在Spider其他挑战性变体上的鲁棒性，并展示了SQL-PaLM卓越的泛化能力。通过大量案例研究，我们还展示了基于LLM的文本到SQL模型令人印象深刻的智能能力及多种成功驱动因素。