One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases. For the task of converting natural language text to SQL queries, Text-to-SQL, adaptation of LLMs is of paramount importance, both in in-context learning and fine-tuning settings, depending on the amount of adaptation data used. In this paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is based on an execution-based self-consistency prompting approach designed for Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our best knowledge is the first to outperform previous state-of-the-art with fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying SQL-PaLM to real-world scenarios we further evaluate its robustness on other challenging variants of Spider and demonstrate the superior generalization capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate the impressive intelligent capabilities and various success enablers of LLM-based Text-to-SQL.

翻译：大型语言模型（LLM）的一个显著涌现能力是生成代码，包括数据库的结构化查询语言（SQL）。对于将自然语言文本转换为SQL查询的任务（即Text-to-SQL），LLM的适应性在上下文学习和微调两种设置中均至关重要，具体取决于所使用的适应性数据量。本文提出了一种基于LLM的Text-to-SQL模型SQL-PaLM，利用PaLM-2在两种设置下均推动了技术前沿。少样本SQL-PaLM基于专为Text-to-SQL设计的执行一致性提示方法，在Spider数据集上达到了77.3%的测试集准确率，据我们所知，这是首次以显著优势（4%）超越此前基于微调的最先进方法。此外，我们证明微调后的SQL-PaLM性能进一步提升1%。为将SQL-PaLM应用于实际场景，我们进一步评估了其在Spider其他挑战性变体上的鲁棒性，并展示了SQL-PaLM卓越的泛化能力。此外，通过大量案例研究，我们展示了基于LLM的Text-to-SQL令人印象深刻的智能能力及多种成功要素。