Detecting structural similarity between queries is essential for selecting examples in in-context learning models. However, assessing structural similarity based solely on the natural language expressions of queries, without considering SQL queries, presents a significant challenge. This paper explores the significance of this similarity metric and proposes a model for accurately estimating it. To achieve this, we leverage a dataset comprising 170k question pairs, meticulously curated to train a similarity prediction model. Our comprehensive evaluation demonstrates that the proposed model adeptly captures the structural similarity between questions, as evidenced by improvements in Kendall-Tau distance and precision@k metrics. Notably, our model outperforms strong competitive embedding models from OpenAI and Cohere. Furthermore, compared to these competitive models, our proposed encoder enhances the downstream performance of NL2SQL models in 1-shot in-context learning scenarios by 1-2\% for GPT-3.5-turbo, 4-8\% for CodeLlama-7B, and 2-3\% for CodeLlama-13B.

翻译：查询之间的结构相似性检测对于在上下文学习模型中选择示例至关重要。然而，仅基于查询的自然语言表达而不考虑SQL查询来评估结构相似性是一项重大挑战。本文探讨了该相似性度量的重要性，并提出了一种精确估计该度量的模型。为实现这一目标，我们利用包含17万对问题对的精心筛选数据集来训练相似性预测模型。全面评估表明，所提模型能够有效捕捉问题之间的结构相似性，这在Kendall-Tau距离和precision@k指标上的改进中得到验证。值得注意的是，我们的模型优于OpenAI和Cohere的强竞争性嵌入模型。此外，与这些竞争模型相比，我们提出的编码器在一次性上下文学习场景中，将下游NL2SQL模型的性能提升了GPT-3.5-turbo的1-2%、CodeLlama-7B的4-8%以及CodeLlama-13B的2-3%。