Large language models (LLMs) have demonstrated great success in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge database to augment LLMs, makes up those drawbacks of LLMs. This paper reviews all significant techniques of RAG, especially in the retriever and the retrieval fusions. Besides, tutorial codes are provided for implementing the representative techniques in RAG. This paper further discusses the RAG training, including RAG with/without datastore update. Then, we introduce the application of RAG in representative natural language processing tasks and industrial scenarios. Finally, this paper discusses the future directions and challenges of RAG for promoting its development.

翻译：大型语言模型（LLM）凭借其存储知识的海量参数，已在多个领域展现出卓越性能。然而，LLM仍存在若干关键问题，例如幻觉现象、知识更新困难以及缺乏领域专业知识。检索增强生成（RAG）通过利用外部知识库增强LLM，有效弥补了这些缺陷。本文系统综述了RAG的核心技术，重点聚焦于检索器与检索融合机制。同时，为RAG典型技术的实现提供了教程代码。本文进一步探讨了RAG的训练范式，包括数据存储更新与静态存储两种模式。随后，我们阐述了RAG在典型自然语言处理任务及工业场景中的应用。最后，本文讨论了RAG的未来发展方向与挑战，以推动该领域的持续发展。