Retrieval-Augmented Generation (RAG) is essential for integrating external knowledge into Large Language Model (LLM) outputs. While the literature on RAG is growing, it primarily focuses on systematic reviews and comparisons of new state-of-the-art (SoTA) techniques against their predecessors, with a gap in extensive experimental comparisons. This study begins to address this gap by assessing various RAG methods' impacts on retrieval precision and answer similarity. We found that Hypothetical Document Embedding (HyDE) and LLM reranking significantly enhance retrieval precision. However, Maximal Marginal Relevance (MMR) and Cohere rerank did not exhibit notable advantages over a baseline Naive RAG system, and Multi-query approaches underperformed. Sentence Window Retrieval emerged as the most effective for retrieval precision, despite its variable performance on answer similarity. The study confirms the potential of the Document Summary Index as a competent retrieval approach. All resources related to this research are publicly accessible for further investigation through our GitHub repository ARAGOG (https://github.com/predlico/ARAGOG). We welcome the community to further this exploratory study in RAG systems.

翻译：检索增强生成（RAG）对于将外部知识整合到大语言模型（LLM）输出中至关重要。尽管关于RAG的文献日益增多，但主要集中在系统性综述以及新型前沿（SoTA）技术与其前身方法的比较上，缺乏广泛的实验对比研究。本研究通过评估不同RAG方法对检索精度和答案相似度的影响，初步填补了这一空白。我们发现假设文档嵌入（HyDE）和LLM重排序显著提升了检索精度。然而，最大边际相关性（MMR）和Cohere重排序相较于基线朴素RAG系统未展现出明显优势，而多查询方法的表现则较差。句窗检索在检索精度方面最为有效，尽管其在答案相似度上的表现存在波动。本研究证实了文档摘要索引作为有效检索方法的潜力。与此研究相关的所有资源均通过我们的GitHub仓库ARAGOG（https://github.com/predlico/ARAGOG）公开提供，供进一步研究使用。我们欢迎学界共同推进这一关于RAG系统的探索性研究。