Several data warehouse and database providers have recently introduced extensions to SQL called AI Queries, enabling users to specify functions and conditions in SQL that are evaluated by LLMs, thereby broadening significantly the kinds of queries one can express over the combination of structured and unstructured data. LLMs offer remarkable semantic reasoning capabilities, making them an essential tool for complex and nuanced queries that blend structured and unstructured data. While extremely powerful, these AI queries can become prohibitively costly when invoked thousands of times. This paper provides an extensive evaluation of a recent AI query approximation approach that enables low cost analytics and database applications to benefit from AI queries. The approach delivers >100x cost and latency reduction for the semantic filter operator and also important gains for semantic ranking. The cost and performance gains come from utilizing cheap and accurate proxy models over embedding vectors. We show that despite the massive gains in latency and cost, these proxy models preserve accuracy and occasionally improve accuracy across various benchmark datasets, including the extended Amazon reviews benchmark that has 10M rows. We present an OLAP-friendly architecture within Google BigQuery for this approach for purely online (ad hoc) queries, and a low-latency HTAP database-friendly architecture in AlloyDB that could further improve the latency by moving the proxy model training offline. We present techniques that accelerate the proxy model training.

翻译：多家数据仓库和数据库提供商近期推出了名为AI查询的SQL扩展功能，允许用户在SQL中指定由大语言模型（LLM）评估的函数和条件，从而极大拓展了在结构化与非结构化数据组合上可表达的查询类型。LLM具备卓越的语义推理能力，使其成为处理融合结构化与非结构化数据的复杂精细查询的关键工具。尽管功能强大，但当这些AI查询被调用数千次时，成本可能高得令人望而却步。本文对一种新型AI查询近似方法进行了全面评估，该方法使低成本的分析和数据库应用能够受益于AI查询。该方案为语义过滤操作符实现了超过100倍的成本与延迟降低，并为语义排序带来了重要改进。其成本与性能优势源于在嵌入向量上使用廉价且精准的代理模型。我们证明，尽管在延迟和成本上取得了巨大提升，这些代理模型仍能保持准确性，甚至在多个基准数据集（包括包含1000万行的扩展版Amazon评论基准）上偶尔提升准确性。我们在Google BigQuery中为此方法设计了适用于纯在线（即席）查询的OLAP友好型架构，以及在AlloyDB中构建了低延迟HTAP数据库友好型架构——通过将代理模型训练移至离线可进一步优化延迟。我们还提出了加速代理模型训练的技术方案。