Several data warehouse and database providers have recently introduced extensions to SQL called AI Queries, enabling users to specify functions and conditions in SQL that are evaluated by LLMs, thereby broadening significantly the kinds of queries one can express over the combination of structured and unstructured data. LLMs offer remarkable semantic reasoning capabilities, making them an essential tool for complex and nuanced queries that blend structured and unstructured data. While extremely powerful, these AI queries can become prohibitively costly when invoked thousands of times. This paper provides an extensive evaluation of a recent AI query approximation approach that enables low cost analytics and database applications to benefit from AI queries. The approach delivers >100x cost and latency reduction for the semantic filter operator and also important gains for semantic ranking. The cost and performance gains come from utilizing cheap and accurate proxy models over embedding vectors. We show that despite the massive gains in latency and cost, these proxy models preserve accuracy and occasionally improve accuracy across various benchmark datasets, including the extended Amazon reviews benchmark that has 10M rows. We present an OLAP-friendly architecture within Google BigQuery for this approach for purely online (ad hoc) queries, and a low-latency HTAP database-friendly architecture in AlloyDB that could further improve the latency by moving the proxy model training offline. We present techniques that accelerate the proxy model training.

翻译：多家数据仓库和数据库提供商近期推出了名为AI查询的SQL扩展功能，支持用户在SQL中定义由大语言模型评估的函数和条件，从而显著拓展了可对结构化与非结构化数据组合进行查询的种类。大语言模型展现出卓越的语义推理能力，使其成为处理融合结构化与非结构化数据的复杂精细查询的关键工具。尽管功能极其强大，当此类AI查询被调用数千次时，其成本可能变得高得令人望而却步。本文对近期提出的一种AI查询近似方法进行了全面评估，该方法使低成本的分析与数据库应用能够受益于AI查询。该方法为语义过滤操作符实现了超过100倍的成本与延迟降低，并在语义排序方面也取得了重要改进。其成本与性能优势源于利用基于嵌入向量的廉价且准确的代理模型。我们证明，尽管延迟和成本大幅降低，这些代理模型仍能保持准确性，甚至在包括包含1000万行记录的扩展版亚马逊评论基准在内的多个基准数据集上偶尔提升准确性。我们针对纯在线查询在Google BigQuery中提出了支持该方法的OLAP友好型架构，以及在AlloyDB中开发了低延迟的HTAP数据库友好型架构，后者可通过将代理模型训练移至离线进一步改善延迟。此外，我们提出了加速代理模型训练的技术。