One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases. For the task of converting natural language text to SQL queries, Text-to-SQL, adaptation of LLMs is of paramount importance, both in in-context learning and fine-tuning settings, depending on the amount of adaptation data used. In this paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is based on an execution-based self-consistency prompting approach designed for Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our best knowledge is the first to outperform previous state-of-the-art with fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying SQL-PaLM to real-world scenarios we further evaluate its robustness on other challenging variants of Spider and demonstrate the superior generalization capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate the impressive intelligent capabilities and various success enablers of LLM-based Text-to-SQL.

翻译：摘要：大语言模型的一项令人瞩目的涌现能力是生成代码，包括数据库的结构化查询语言（SQL）。对于将自然语言文本转化为SQL查询的任务（即文本到SQL），大语言模型的适配至关重要，这既包括上下文学习场景，也包括微调场景，具体取决于所使用的适配数据量。本文提出了一种基于大语言模型的文本到SQL模型SQL-PaLM，该模型以PaLM-2为基础，在两个场景下均达到了当前最优水平。少样本SQL-PaLM采用了一种针对文本到SQL设计的基于执行的自我一致性提示方法，在Spider数据集上的测试套件准确率达到77.3%。据我们所知，这是首个显著超越先前基于微调的当前最优方法——且以4%的绝对优势——的模型。此外，我们证明了微调后的SQL-PaLM性能进一步提升1%。为将SQL-PaLM应用于实际场景，我们进一步评估了其在Spider其他具有挑战性变体上的鲁棒性，并展示了SQL-PaLM的卓越泛化能力。此外，通过大量案例研究，我们证明了基于大语言模型的文本到SQL所具备的惊人智能能力及多种成功要素。