Query recommendation systems are ubiquitous in modern search engines, assisting users in producing effective queries to meet their information needs. However, these systems require a large amount of data to produce good recommendations, such as a large collection of documents to index and query logs. In particular, query logs and user data are not available in cold start scenarios. Query logs are expensive to collect and maintain and require complex and time-consuming cascading pipelines for creating, combining, and ranking recommendations. To address these issues, we frame the query recommendation problem as a generative task, proposing a novel approach called Generative Query Recommendation (GQR). GQR uses an LLM as its foundation and does not require to be trained or fine-tuned to tackle the query recommendation problem. We design a prompt that enables the LLM to understand the specific recommendation task, even using a single example. We then improved our system by proposing a version that exploits query logs called Retriever-Augmented GQR (RA-GQR). RA-GQr dynamically composes its prompt by retrieving similar queries from query logs. GQR approaches reuses a pre-existing neural architecture resulting in a simpler and more ready-to-market approach, even in a cold start scenario. Our proposed GQR obtains state-of-the-art performance in terms of NDCG@10 and clarity score against two commercial search engines and the previous state-of-the-art approach on the Robust04 and ClueWeb09B collections, improving on average the NDCG@10 performance up to ~4% on Robust04 and ClueWeb09B w.r.t the previous best competitor. RA-GQR further improve the NDCG@10 obtaining an increase of ~11%, ~6\% on Robust04 and ClueWeb09B w.r.t the best competitor. Furthermore, our system obtained ~59% of user preferences in a blind user study, proving that our method produces the most engaging queries.
翻译:查询推荐系统在现代搜索引擎中无处不在,旨在协助用户生成有效查询以满足其信息需求。然而,这些系统需要大量数据才能产生优质推荐,例如用于索引的大规模文档集和查询日志。特别是在冷启动场景中,查询日志和用户数据往往无法获取。查询日志的收集与维护成本高昂,且需要复杂耗时的级联流水线来创建、组合和排序推荐。为解决这些问题,我们将查询推荐问题构建为生成式任务,提出了一种称为生成式查询推荐(GQR)的新方法。GQR以大语言模型(LLM)为基础,无需针对查询推荐问题进行训练或微调。我们设计了一种提示模板,使LLM能够理解特定推荐任务,即使仅使用单个示例亦可实现。随后,我们通过提出利用查询日志的增强版本——检索增强型GQR(RA-GQR)来改进系统。RA-GQR通过从查询日志中检索相似查询来动态构建提示。GQR方法复用了已有的神经架构,从而形成更简洁、更易于市场化的解决方案,即使在冷启动场景中亦能适用。我们提出的GQR在Robust04和ClueWeb09B数据集上,针对两个商业搜索引擎及先前最优方法,在NDCG@10和清晰度得分方面均达到最先进性能:相较于先前最佳竞争对手,在Robust04和ClueWeb09B上的NDCG@10平均提升约4%。RA-GQR进一步将NDCG@10提升约11%(Robust04)和约6%(ClueWeb09B)。此外,在盲测用户研究中,我们的系统获得了约59%的用户偏好,证明本方法生成的查询最具吸引力。