Text-to-SQL and Big Data are both extensively benchmarked fields, yet there is limited research that evaluates them jointly. In the real world, Text-to-SQL systems are often embedded with Big Data workflows, such as large-scale data processing or interactive data analytics. We refer to this as ``Text-to-Big SQL''. However, existing text-to-SQL benchmarks remain narrowly scoped and overlook the cost and performance implications that arise at scale. For instance, translation errors that are minor on small datasets lead to substantial cost and latency overheads as data scales, a relevant issue completely ignored by text-to-SQL metrics. In this paper, we overcome this overlooked challenge by introducing novel and representative metrics for evaluating Text-to-Big SQL. Our study focuses on production-level LLM agents, a database-agnostic system adaptable to diverse user needs. Via an extensive evaluation of frontier models, we show that text-to-SQL metrics are insufficient for Big Data. In contrast, our proposed text-to-Big SQL metrics accurately reflect execution efficiency, cost, and the impact of data scale. For example, GPT-4o compensates for roughly 7% lower accuracy than the top-performing later-generation models with up to a 12.16x speedup, while GPT-5.2 is more than twice as cost-effective as Gemini 3 Pro at large input scales.

翻译：文本到SQL和大数据都是广泛研究的热点领域，但将两者联合评估的研究却极为有限。在实际应用中，文本到SQL系统常嵌入大数据工作流，例如大规模数据处理或交互式数据分析等场景。我们将其称为“文本到大规模SQL”。然而，现有文本到SQL基准测试范围过于狭窄，忽视了规模扩展带来的成本与性能影响。例如，在小数据集上微不足道的翻译错误，随着数据规模增大将导致显著的成本和延迟开销——这一关键问题完全被文本到SQL评估指标所忽略。本文通过引入新颖且具代表性的评估指标来攻克这一被忽视的挑战，致力于评估文本到大规模SQL任务。本研究聚焦生产级LLM智能体——一种可适配多样化用户需求的数据库无关系统。通过对前沿模型的大规模评估，我们证明文本到SQL指标不足以应对大数据场景。相较之下，我们提出的文本到大规模SQL指标能准确反映执行效率、成本及数据规模的影响。例如，GPT-4o虽然准确率比表现最优的后代模型低约7%，但其速度提升可达12.16倍；而在大规模输入场景下，GPT-5.2的成本效益比Gemini 3 Pro高出两倍以上。