Recently, code-oriented large language models (LLMs) have demonstrated strong capabilities in translating natural language into executable code. Text-to-SQL is a significant application of this ability, enabling non-technical users to interact with relational databases using natural language. However, state-of-the-art models continue to struggle with highly complex logic, particularly deeply nested statements involving multiple joins and conditions, as well as with real-world database schemas that are noisy or poorly structured. In this paper, we investigate whether curriculum learning can improve the performance of code-based LLMs on Text-to-SQL tasks. Employing benchmarks including Spider and BIRD, we fine-tune models under different curriculum strategies. Our experiments show that naive curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting. To overcome this, we propose a Modular Adapter Composition (MAC) strategy. By sequentially training tier-specific adapters on incremental complexity levels (Easy to Extra-Hard), we create a scaffolded learning environment that improves performance on complex queries. Our approach not only produces measurable performance gains on the Spider and BIRD benchmarks but also provides a flexible, "Lego-like" architecture, allowing models to be composed and deployed based on specific schema difficulty requirements. These findings demonstrate that structured, modular learning is a superior alternative to monolithic fine-tuning for mastering the syntax and logic of complex code generation.
翻译:近期,面向代码的大型语言模型在将自然语言转化为可执行代码方面展现出强大能力。Text-to-SQL作为这一能力的重要应用,使非技术用户能够通过自然语言与关系型数据库交互。然而,当前最先进的模型在处理高度复杂的逻辑(特别是涉及多重连接和条件的深层嵌套语句)以及现实数据库中存在的噪声或结构不良的模式时仍面临挑战。本文探究了课程学习能否提升基于代码的大语言模型在Text-to-SQL任务中的表现。我们采用Spider和BIRD等基准,在不同课程策略下对模型进行微调。实验表明,朴素课程(即仅按复杂度对单轮训练样本排序)因灾难性遗忘而无法超越标准微调。为此,我们提出模块化适配器组合(MAC)策略。通过在递增复杂度层级(从简单到极难)上依次训练层级专用适配器,我们构建了阶梯式学习环境,从而提升了复杂查询的性能。该方法不仅在Spider和BIRD基准上取得了可量化的性能提升,还提供了灵活的“乐高式”架构,使模型能够根据具体模式难度需求进行组合与部署。这些发现表明,结构化的模块化学习是掌握复杂代码生成语法与逻辑的优于整体微调的方案。