While search is the predominant method of accessing information, formulating effective queries remains a challenging task, especially for situations where the users are not familiar with a domain, or searching for documents in other languages, or looking for complex information such as events, which are not easily expressible as queries. Providing example documents or passages of interest, might be easier for a user, however, such query-by-example scenarios are prone to concept drift, and are highly sensitive to the query generation method. This demo illustrates complementary approaches of using LLMs interactively, assisting and enabling the user to provide edits and feedback at all stages of the query formulation process. The proposed Query Generation Assistant is a novel search interface which supports automatic and interactive query generation over a mono-linguial or multi-lingual document collection. Specifically, the proposed assistive interface enables the users to refine the queries generated by different LLMs, to provide feedback on the retrieved documents or passages, and is able to incorporate the users' feedback as prompts to generate more effective queries. The proposed interface is a valuable experimental tool for exploring fine-tuning and prompting of LLMs for query generation to qualitatively evaluate the effectiveness of retrieval and ranking models, and for conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries without such assistance.
翻译:尽管搜索是获取信息的主要方式,但构建有效的查询仍是一项具有挑战性的任务,特别是在用户不熟悉某个领域、跨语言搜索文档或寻找难以用查询表达的事件等复杂信息时。提供感兴趣的示例文档或段落对用户而言可能更为便捷,但这种以示例为查询的方式容易产生概念漂移,且对查询生成方法高度敏感。本演示展示了交互式使用大语言模型(LLM)的补充方法,在查询构建的各个阶段协助用户进行编辑和反馈。所提出的查询生成助手是一种新颖的搜索界面,支持在单语或多语文档集上进行自动化和交互式查询生成。具体而言,该辅助界面使用户能够优化不同LLM生成的查询、对检索到的文档或段落提供反馈,并将用户反馈作为提示融入,以生成更有效的查询。该界面作为一种有价值的实验工具,可用于探索针对查询生成的LLM微调与提示策略、定性评估检索和排序模型的有效性,以及在用户难以自主构建查询的复杂搜索任务中开展人在回路(HITL)实验。