Text2SQL, the task of generating SQL queries from natural language text, is a critical challenge in data engineering. Recently, Large Language Models (LLMs) have demonstrated superior performance for this task due to their advanced comprehension and generation capabilities. However, privacy and cost considerations prevent companies from using Text2SQL solutions based on external LLMs offered as a service. Rather, small LLMs (SLMs) that are openly available and can hosted in-house are adopted. These SLMs, in turn, lack the generalization capabilities of larger LLMs, which impairs their effectiveness for complex tasks such as Text2SQL. To address these limitations, we propose MATS, a novel Text2SQL framework designed specifically for SLMs. MATS uses a multi-agent mechanism that assigns specialized roles to auxiliary agents, reducing individual workloads and fostering interaction. A training scheme based on reinforcement learning aligns these agents using feedback obtained during execution, thereby maintaining competitive performance despite a limited LLM size. Evaluation results using on benchmark datasets show that MATS, deployed on a single- GPU server, yields accuracy that are on-par with large-scale LLMs when using significantly fewer parameters. Our source code and data are available at https://github.com/thanhdath/mats-sql.
翻译:Text2SQL,即从自然语言文本生成SQL查询的任务,是数据工程领域的一项关键挑战。近年来,大型语言模型(LLMs)凭借其先进的理解与生成能力,在该任务上展现出卓越性能。然而,出于隐私与成本考量,企业难以采用基于外部服务化LLMs的Text2SQL解决方案。因此,业界转而采用可公开获取且能内部部署的小型语言模型(SLMs)。但这些SLMs缺乏大型LLMs的泛化能力,导致其在处理如Text2SQL等复杂任务时效果受限。为应对这些不足,我们提出了MATS——一个专为SLMs设计的新型Text2SQL框架。MATS采用多智能体机制,为辅助智能体分配专门角色,以降低个体工作负荷并促进交互协作。通过基于强化学习的训练方案,利用执行过程中获得的反馈对这些智能体进行对齐,从而在有限模型规模下保持竞争力。在基准数据集上的评估结果表明,部署于单GPU服务器的MATS仅使用显著更少的参数,即可达到与大规模LLMs相当的准确率。我们的源代码与数据公开于https://github.com/thanhdath/mats-sql。