Text-to-SQL is a key natural language processing task that maps natural language questions to SQL queries, enabling intuitive interaction with web-based databases. Although current methods perform well on benchmarks like BIRD and Spider, they struggle with complex reasoning, domain knowledge, and hypothetical queries, and remain costly in enterprise deployment. To address these issues, we propose a framework named IESR(Information Enhanced Structured Reasoning) for lightweight large language models: (i) leverages LLMs for key information understanding and schema linking, and decoupling mathematical computation and SQL generation, (ii) integrates a multi-path reasoning mechanism based on Monte Carlo Tree Search (MCTS) with majority voting, and (iii) introduces a trajectory consistency verification module with a discriminator model to ensure accuracy and consistency. Experimental results demonstrate that IESR achieves state-of-the-art performance on the complex reasoning benchmark LogicCat (24.28 EX) and the Archer dataset (37.28 EX) using only compact lightweight models without fine-tuning. Furthermore, our analysis reveals that current coder models exhibit notable biases and deficiencies in physical knowledge, mathematical computation, and common-sense reasoning, highlighting important directions for future research. We released code at https://github.com/Ffunkytao/IESR-SLM.
翻译:文本到SQL是一项关键的自然语言处理任务,它将自然语言问题映射为SQL查询,从而实现与基于网络数据库的直观交互。尽管现有方法在BIRD和Spider等基准测试中表现良好,但在复杂推理、领域知识和假设性查询方面仍存在困难,且在企业部署中成本高昂。为解决这些问题,我们提出了一个名为IESR(信息增强结构化推理)的轻量化大语言模型框架:(i)利用大语言模型进行关键信息理解和模式链接,并将数学计算与SQL生成解耦;(ii)整合了基于蒙特卡洛树搜索(MCTS)与多数投票机制的多路径推理机制;(iii)引入了一个带有判别器模型的轨迹一致性验证模块,以确保准确性和一致性。实验结果表明,IESR仅使用紧凑的轻量化模型且无需微调,即在复杂推理基准LogicCat(24.28 EX)和Archer数据集(37.28 EX)上达到了最先进的性能。此外,我们的分析揭示了当前编码器模型在物理知识、数学计算和常识推理方面存在显著的偏见和不足,这为未来研究指明了重要方向。我们在https://github.com/Ffunkytao/IESR-SLM发布了代码。