This paper presents our approach to the TREC Interactive Knowledge Assistance Track (iKAT), which focuses on improving conversational information-seeking (CIS) systems. While recent advancements in CIS have improved conversational agents' ability to assist users, significant challenges remain in understanding context and retrieving relevant documents across domains and dialogue turns. To address these issues, we extend the Generate-Retrieve-Generate pipeline by developing passage queries (PQs) that align with the target document's expected format to improve query-document matching during retrieval. We propose two variations of this approach: Weighted Reranking and Short and Long Passages. Each method leverages a Meta Llama model for context understanding and generating queries and responses. Passage ranking evaluation results show that the Short and Long Passages approach outperformed the organizers' baselines, performed best among Llama-based systems in the track, and achieved results comparable to GPT-4-based systems. These results indicate that the method effectively balances efficiency and performance. Findings suggest that PQs improve semantic alignment with target documents and demonstrate their potential to improve multi-turn dialogue systems.
翻译:本文介绍了我们在TREC交互式知识辅助赛道(iKAT)中的研究方案,该赛道专注于改进对话式信息检索系统。尽管对话式信息检索领域的最新进展提升了对话代理协助用户的能力,但在跨领域、跨对话轮次的上下文理解与相关文档检索方面仍存在显著挑战。为应对这些问题,我们通过开发与目标文档预期格式对齐的段落查询来扩展生成-检索-生成流程,以提升检索过程中的查询-文档匹配度。我们提出该方法的两种变体:加权重排序与长短段落策略。每种方法均利用Meta Llama模型进行上下文理解以及查询与响应的生成。段落排序评估结果表明:长短段落策略的表现超越了组织方设定的基线,在该赛道所有基于Llama的系统中性能最优,且取得了与基于GPT-4系统相当的结果。这些发现表明该方法在效率与性能间实现了有效平衡。研究结果证实段落查询能提升与目标文档的语义对齐度,并展现了其改进多轮对话系统的潜力。