In the era of large language models, Text-to-SQL, as a natural language interface for databases, is playing an increasingly important role. The sota Text-to-SQL models have achieved impressive accuracy, but their performance critically relies on expert-written evidence, which typically clarifies schema and value linking that existing models struggle to identify. Such limitations stem from the ambiguity of user queries and, more importantly, the complexity of comprehending large-scale and dynamic database values. Consequently, in real-world scenarios where expert assistance is unavailable, existing methods suffer a severe performance collapse, with execution accuracy dropping by over 10%. This underscores their lack of robustness. To address this, we propose DIVER, a robust system that automates evidence reasoning with dynamic interactive value linking. It leverages a compatible toolbox containing diverse tools to probe the database. Then, restricted by a structured workspace (CoTF, Chain of Thoughts and Facts), it reflects based on probe results and selects a new tool for next round of probing. Through this automatically iterative process, DIVER identifies schema and value linking missed by existing methods. Based on these accurate linkings, DIVER is able to infer correct usage of SQL functions and formulas and generate high-quality evidence, achieving robust Text-to-SQL without expert assistance. Extensive experiments demonstrate that: 1) The DIVER system significantly enhances the robustness of various Text-to-SQL models, improving performance by up to 10.82% in Execution Accuracy (EX) and 16.09% in Valid Efficiency Score (VES). 2) Our dynamic interactive value linking significantly improves the robustness of existing systems and the accuracy of schema and value linking, especially when confronted with challenges posed by large-scale, dynamic database values.
翻译:在大语言模型时代,文本到SQL作为数据库的自然语言接口正发挥着日益重要的作用。当前最先进的文本到SQL模型已取得令人瞩目的准确率,但其性能严重依赖于专家编写的证据,这些证据通常用于澄清现有模型难以识别的模式与值链接。此类局限性源于用户查询的模糊性,更重要的是理解大规模动态数据库值的复杂性。因此,在缺乏专家协助的实际应用场景中,现有方法会遭遇严重的性能崩溃,执行准确率下降超过10%,这凸显了其鲁棒性的不足。为解决这一问题,我们提出DIVER系统——一种通过动态交互式值链接实现证据推理自动化的鲁棒性系统。该系统利用包含多样化工具的兼容工具箱对数据库进行探查。随后,在结构化工作空间(CoTF,即思维与事实链)的约束下,系统基于探查结果进行反思,并选择新工具进行下一轮探查。通过这种自动迭代过程,DIVER能够识别出现有方法遗漏的模式与值链接。基于这些精确链接,DIVER得以推断SQL函数与公式的正确用法,并生成高质量证据,从而在无需专家协助的情况下实现鲁棒的文本到SQL。大量实验表明:1)DIVER系统显著提升了多种文本到SQL模型的鲁棒性,在执行准确率(EX)上最高提升10.82%,在有效效率分数(VES)上最高提升16.09%。2)我们提出的动态交互式值链接机制显著增强了现有系统的鲁棒性及模式与值链接的准确性,尤其是在应对大规模动态数据库值带来的挑战时表现突出。