Text-to-SQL models can generate a list of candidate SQL queries, and the best query is often in the candidate list, but not at the top of the list. An effective re-rank method can select the right SQL query from the candidate list and improve the model's performance. Previous studies on code generation automatically generate test cases and use them to re-rank candidate codes. However, automatic test case generation for text-to-SQL is an understudied field. We propose an automatic test case generation method that first generates a database and then uses LLMs to predict the ground truth, which is the expected execution results of the ground truth SQL query on this database. To reduce the difficulty for LLMs to predict, we conduct experiments to search for ways to generate easy databases for LLMs and design easy-to-understand prompts. Based on our test case generation method, we propose a re-rank method to select the right SQL query from the candidate list. Given a candidate list, our method can generate test cases and re-rank the candidate list according to their pass numbers on these test cases and their generation probabilities. The experiment results on the validation dataset of Spider show that the performance of some state-of-the-art models can get a 3.6\% improvement after applying our re-rank method.

翻译：文本到SQL模型可生成候选SQL查询列表，而最佳查询通常位于候选列表中但非首位。有效的重排序方法能从候选列表中选取正确SQL查询并提升模型性能。先前关于代码生成的研究通过自动生成测试用例对候选代码进行重排序，然而文本到SQL领域的自动测试用例生成尚属未充分研究领域。我们提出一种自动测试用例生成方法：首先生成数据库，再利用大语言模型预测真实标签（即在该数据库上执行真实SQL查询的预期结果）。为降低LLM的预测难度，我们通过实验探索易处理数据库的生成策略，并设计易于理解的提示词。基于该测试用例生成方法，我们提出从候选列表中选取正确SQL查询的重排序方法：给定候选列表，该方法能生成测试用例，并根据候选查询在这些测试用例上的通过数量及生成概率进行重排序。在Spider验证集上的实验表明，采用我们的重排序方法后，部分最优模型的性能可提升3.6%。