Retrieval-augmented large language models (R-LLMs) combine pre-trained large language models (LLMs) with information retrieval systems to improve the accuracy of factual question-answering. However, current libraries for building R-LLMs provide high-level abstractions without sufficient transparency for evaluating and optimizing prompts within specific inference processes such as retrieval and generation. To address this gap, we present RaLLe, an open-source framework designed to facilitate the development, evaluation, and optimization of R-LLMs for knowledge-intensive tasks. With RaLLe, developers can easily develop and evaluate R-LLMs, improving hand-crafted prompts, assessing individual inference processes, and objectively measuring overall system performance quantitatively. By leveraging these features, developers can enhance the performance and accuracy of their R-LLMs in knowledge-intensive generation tasks. We open-source our code at https://github.com/yhoshi3/RaLLe.

翻译：检索增强型大语言模型（R-LLMs）将预训练的大语言模型（LLMs）与信息检索系统相结合，以提高事实性问答的准确性。然而，当前用于构建R-LLMs的库仅提供高层抽象，缺乏足够的透明度来评估和优化检索与生成等特定推理过程中的提示。为解决这一问题，我们提出了RaLLe——一个开源框架，旨在促进面向知识密集型任务的R-LLMs的开发、评估与优化。借助RaLLe，开发者能够轻松开发与评估R-LLMs，改进手工设计的提示，评估各个推理过程，并客观定量衡量整体系统性能。通过利用这些特性，开发者可以提升其R-LLMs在知识密集型生成任务中的性能与准确性。我们将代码开源在https://github.com/yhoshi3/RaLLe。