Retrieval-Augmented Generation (RAG) is sensitive to the vast hyperparameters of the retriever and generator, yet optimizing them using given queries is a challenging task due to the complex interactions and expensive evaluation costs. Existing algorithms are ineffective and slow in convergence, since they often treat RAG as a monolithic black box or only optimize partial hyperparameters. In this paper, we propose CDS4RAG, a framework that optimizes the full RAG hyperparameters using given queries via a new cyclic dual-sequential formulation. CDS4RAG is special in the sense that it distinguishes the hyperparameters of the retriever and generator, cyclically optimizing them in turn. Such a paradigm allows us to design fine-grained within-cycle budget provision and expedite the optimization via cross-cycle seeding when optimizing the generator. CDS4RAG is also an algorithm-agnostic framework that can be paired with diverse general algorithms. Through experiments on four common benchmarks and two backbone LLMs, we reveal that CDS4RAG considerably boosts the vanilla algorithms in 21/24 cases while significantly outperforming state-of-the-art algorithms in all cases with up to 1.54x improvements of generation quality and better speedup.

翻译：检索增强生成（RAG）对检索器和生成器的大量超参数极为敏感，但由于复杂的交互作用和昂贵的评估成本，利用给定查询优化这些超参数是一项具有挑战性的任务。现有算法往往将RAG视为单一黑盒或仅优化部分超参数，导致效率低下且收敛缓慢。本文提出CDS4RAG框架，该框架通过一种新颖的循环双序列公式，利用给定查询优化完整的RAG超参数。CDS4RAG的特殊之处在于它区分了检索器和生成器的超参数，并轮流对其进行循环优化。这种范式使得我们能够设计细粒度的周期内预算分配，并在优化生成器时通过跨周期播种加速优化进程。CDS4RAG还是一个算法无关框架，可与多种通用算法配对使用。通过在四个常用基准测试和两个骨干大语言模型上的实验，我们揭示CDS4RAG在21/24个案例中显著提升了原始算法性能，并在所有案例中均大幅超越现有最优算法，生成质量提升高达1.54倍且加速效果更优。