Retrieval Augmented Generation (RAG) has emerged as an effective solution for mitigating hallucinations in Large Language Models (LLMs). The retrieval stage in RAG typically involves a pre-trained embedding model, which converts queries and passages into vectors to capture their semantics. However, a standard pre-trained embedding model may exhibit sub-optimal performance when applied to specific domain knowledge, necessitating fine-tuning. This paper addresses scenarios where the embeddings are only available from a black-box model. We introduce Model augmented fine-tuning (Mafin) -- a novel approach for fine-tuning a black-box embedding model by augmenting it with a trainable embedding model. Our results demonstrate that Mafin significantly enhances the performance of the black-box embeddings by only requiring the training of a small augmented model. We validate the effectiveness of our method on both labeled and unlabeled datasets, illustrating its broad applicability and efficiency.

翻译：检索增强生成（RAG）已成为缓解大型语言模型（LLMs）幻觉问题的有效方案。RAG的检索阶段通常采用预训练嵌入模型，将查询和文本片段转换为向量以捕获其语义。然而，当应用于特定领域知识时，标准预训练嵌入模型可能表现出次优性能，因此需要微调。本文针对嵌入仅可从黑盒模型获取的场景展开研究。我们提出模型增强微调（Mafin）——一种通过增强可训练嵌入模型来微调黑盒嵌入模型的新方法。结果表明，仅需训练一个轻量级增强模型，Mafin即可显著提升黑盒嵌入性能。我们在标注与非标注数据集上验证了该方法的有效性，充分展现了其广泛的适用性与高效性。