Implementing Retrieval-Augmented Generation (RAG) systems is inherently complex, requiring deep understanding of data, use cases, and intricate design decisions. Additionally, evaluating these systems presents significant challenges, necessitating assessment of both retrieval accuracy and generative quality through a multi-faceted approach. We introduce RAG Foundry, an open-source framework for augmenting large language models for RAG use cases. RAG Foundry integrates data creation, training, inference and evaluation into a single workflow, facilitating the creation of data-augmented datasets for training and evaluating large language models in RAG settings. This integration enables rapid prototyping and experimentation with various RAG techniques, allowing users to easily generate datasets and train RAG models using internal or specialized knowledge sources. We demonstrate the framework effectiveness by augmenting and fine-tuning Llama-3 and Phi-3 models with diverse RAG configurations, showcasing consistent improvements across three knowledge-intensive datasets. Code is released as open-source in https://github.com/IntelLabs/RAGFoundry.

翻译：实现检索增强生成（RAG）系统本质上是复杂的，需要对数据、用例以及复杂的设计决策有深入的理解。此外，评估这些系统也面临重大挑战，需要通过多维度方法同时评估检索准确性和生成质量。我们介绍了RAG Foundry，这是一个用于增强大语言模型以应对RAG用例的开源框架。RAG Foundry将数据创建、训练、推理和评估集成到单一工作流中，便于为RAG场景下训练和评估大语言模型创建数据增强的数据集。这种集成支持对各种RAG技术进行快速原型设计和实验，使用户能够轻松利用内部或专业知识源生成数据集并训练RAG模型。我们通过使用多种RAG配置对Llama-3和Phi-3模型进行增强与微调，展示了该框架的有效性，并在三个知识密集型数据集上取得了一致的性能提升。代码已在 https://github.com/IntelLabs/RAGFoundry 开源发布。