Large language models (LLMS) have shown increasing effectiveness in Text-to-SQL tasks. However, another closely related problem, Cross-System SQL Translation (a.k.a., SQL-to-SQL), which adapts a query written for one database system (e.g., MySQL) into its equivalent one for another system (e.g., ClickHouse), is of great practical importance but remains underexplored. Existing SQL benchmarks are not well-suited for SQL-to-SQL evaluation, which (1) focus on a limited set of database systems (often just SQLite) and (2) cannot capture many system-specific SQL dialects (e.g., customized functions, data types, and syntax rules). Thus, in this paper, we introduce PARROT, a Practical And Realistic BenchmaRk for CrOss-System SQL Translation. PARROT comprises 598 translation pairs from 38 open-source benchmarks and real-world business services, specifically prepared to challenge system-specific SQL understanding (e.g., LLMS achieve lower than 38.53% accuracy on average). We also provide multiple benchmark variants, including PARROT-Diverse with 28,003 translations (for extensive syntax testing) and PARROT-Simple with 5,306 representative samples (for focused stress testing), covering 22 production-grade database systems. To promote future research, we release a public leaderboard and source code at: https://code4db.github.io/parrot-bench/.

翻译：大语言模型在文本到SQL任务中展现出日益增强的性能。然而，另一个密切相关的实际问题——跨系统SQL翻译（亦称SQL到SQL转换）——即将针对某一数据库系统（如MySQL）编写的查询语句适配为另一系统（如ClickHouse）的等效语句，虽具有重要现实意义却仍未得到充分探索。现有SQL基准测试集并不适用于SQL到SQL评估，因其（1）仅涵盖有限数据库系统（通常仅为SQLite），且（2）无法捕捉众多系统特定的SQL方言（例如定制函数、数据类型及语法规则）。为此，本文提出PARROT：一个面向跨系统SQL翻译的实用且真实的基准测试集。PARROT包含来自38个开源基准与真实商业服务的598组翻译对，专门设计用于挑战系统特定的SQL理解能力（例如大语言模型平均准确率低于38.53%）。我们还提供多个基准变体，包括含28,003组翻译的PARROT-Diverse（用于扩展语法测试）与含5,306组代表性样本的PARROT-Simple（用于聚焦压力测试），覆盖22个生产级数据库系统。为促进未来研究，我们已公开排行榜与源代码：https://code4db.github.io/parrot-bench/。