Formulating effective search queries remains a challenging task, particularly when users lack expertise in a specific domain or are not proficient in the language of the content. Providing example documents of interest might be easier for a user. However, such query-by-example scenarios are prone to concept drift, and the retrieval effectiveness is highly sensitive to the query generation method, without a clear way to incorporate user feedback. To enable exploration and to support Human-In-The-Loop experiments we propose QueryExplorer -- an interactive query generation, reformulation, and retrieval interface with support for HuggingFace generation models and PyTerrier's retrieval pipelines and datasets, and extensive logging of human feedback. To allow users to create and modify effective queries, our demo supports complementary approaches of using LLMs interactively, assisting the user with edits and feedback at multiple stages of the query formulation process. With support for recording fine-grained interactions and user annotations, QueryExplorer can serve as a valuable experimental and research platform for annotation, qualitative evaluation, and conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries.
翻译:制定有效的搜索查询仍是一项具有挑战性的任务,特别是当用户缺乏特定领域的专业知识或对内容语言不熟练时。向用户提供感兴趣的示例文档可能更为便捷。然而,这种基于示例的查询场景容易引发概念漂移,且检索效果对查询生成方法高度敏感,且缺乏明确方式融入用户反馈。为支持探索和人机协同实验,我们提出QueryExplorer——一个交互式查询生成、重构与检索界面,支持HuggingFace生成模型、PyTerrier检索流程与数据集,并提供用户反馈的全面日志记录。为帮助用户创建和修改高效查询,我们的演示系统支持交互式使用大语言模型(LLM)的互补方法,在查询制定的多阶段中辅助用户进行编辑与反馈。通过支持细粒度交互记录与用户标注,QueryExplorer可作为注释、定性评估以及针对用户难以制定查询的复杂搜索任务开展人机协同(HITL)实验的宝贵实验与研究平台。