In this work, we introduce ChatQA, a suite of models that outperform GPT-4 on retrieval-augmented generation (RAG) and conversational question answering (QA). To enhance generation, we propose a two-stage instruction tuning method that significantly boosts the performance of RAG. For effective retrieval, we introduce a dense retriever optimized for conversational QA, which yields results comparable to the alternative state-of-the-art query rewriting models, while substantially reducing deployment costs. We also present the ChatRAG Bench, which encompasses ten datasets covering comprehensive evaluations on RAG, table-related QA, arithmetic calculations, and scenarios involving unanswerable questions. Our ChatQA-1.0-70B (score: 54.14), built on Llama2, a weaker foundation model than GPT-4, can slightly outperform GPT-4-0613 (score: 53.90) and GPT-4-Turbo-2024-04-09 (score: 54.03) on the ChatRAG Bench, without relying on any synthetic data from OpenAI GPT models. Notably, Llama3-ChatQA-1.5-70B model surpasses the accuracy of GPT-4-Turbo-2024-04-09 by a margin. To advance research in this field, we open-sourced the model weights, instruction tuning data, ChatRAG Bench, and retriever for the community: https://chatqa-project.github.io/.

翻译：本文介绍了ChatQA模型系列，其在检索增强生成（RAG）与对话式问答（QA）任务上的性能超越了GPT-4。针对生成环节，我们提出了一种两阶段指令微调方法，显著提升了RAG的性能。在检索环节，我们引入了一种为对话式问答优化的稠密检索器，其性能可与当前最先进的查询重写模型相媲美，同时大幅降低了部署成本。我们还构建了ChatRAG基准（ChatRAG Bench），涵盖十个数据集，实现对RAG、表格相关问答、算术计算及不可回答问题的综合评估。基于Llama2（基础能力弱于GPT-4）开发的ChatQA-1.0-70B模型（得分54.14），在ChatRAG基准上的表现略优于GPT-4-0613（得分53.90）和GPT-4-Turbo-2024-04-09（得分54.03），且未使用OpenAI GPT模型的任何合成数据。值得注意的是，Llama3-ChatQA-1.5-70B模型在准确率上显著超越了GPT-4-Turbo-2024-04-09。为推动该领域研究，我们向社区开源了模型权重、指令微调数据、ChatRAG基准及检索器：https://chatqa-project.github.io/。