This study investigates various approaches to using Large Language Models (LLMs) for Text-to-SQL program synthesis, focusing on the outcomes and insights derived. Employing the popular Text-to-SQL dataset, spider, the goal was to input a natural language question along with the database schema and output the correct SQL SELECT query. The initial approach was to fine-tune a local and open-source model to generate the SELECT query. After QLoRa fine-tuning WizardLM's WizardCoder-15B model on the spider dataset, the execution accuracy for generated queries rose to a high of 61%. With the second approach, using the fine-tuned gpt-3.5-turbo-16k (Few-shot) + gpt-4-turbo (Zero-shot error correction), the execution accuracy reached a high of 82.1%. Of all the incorrect queries, most can be categorized into a seven different categories of what went wrong: selecting the wrong columns or wrong order of columns, grouping by the wrong column, predicting the wrong values in conditionals, using different aggregates than the ground truth, extra or too few JOIN clauses, inconsistencies in the Spider dataset, and lastly completely incorrect query structure. Most if not all of the queries fall into these categories and it is insightful to understanding where the faults still lie with LLM program synthesis and where they can be improved.

翻译：本研究探讨了利用大型语言模型（LLMs）进行文本到SQL程序合成的多种方法，重点分析了相关结果与见解。采用广泛使用的文本到SQL数据集Spider，目标是输入自然语言问题及数据库模式，并输出正确的SQL SELECT查询。第一种方法是对本地开源模型进行微调以生成SELECT查询。通过对Spider数据集上的WizardLM的WizardCoder-15B模型进行QLoRa微调，生成查询的执行准确率最高达到61%。第二种方法采用微调后的gpt-3.5-turbo-16k（少样本）结合gpt-4-turbo（零样本错误修正），执行准确率最高达到82.1%。在所有错误查询中，大多可归为七类错误：选择错误列或列顺序错误、分组依据列错误、条件中预测错误值、使用与真实值不同的聚合函数、多出或缺少JOIN子句、Spider数据集中的不一致性，以及完全错误的查询结构。几乎所有查询均落入这些类别，这有助于理解LLM程序合成中仍存在的缺陷及改进方向。