Retrieval-augmented generation supports language models to strengthen their factual groundings by providing external contexts. However, language models often face challenges when given extensive information, diminishing their effectiveness in solving questions. Context compression tackles this issue by filtering out irrelevant information, but current methods still struggle in realistic scenarios where crucial information cannot be captured with a single-step approach. To overcome this limitation, we introduce CompAct, a novel framework that employs an active strategy to condense extensive documents without losing key information. Our experiments demonstrate that CompAct brings significant improvements in both performance and compression rate on multi-hop question-answering (QA) benchmarks. CompAct flexibly operates as a cost-efficient plug-in module with various off-the-shelf retrievers or readers, achieving exceptionally high compression rates (47x).

翻译：检索增强生成通过提供外部上下文来增强语言模型的事实依据。然而，当面对大量信息时，语言模型常面临挑战，其解决问题的有效性会降低。上下文压缩通过过滤无关信息来解决这一问题，但现有方法在现实场景中仍存在局限，即关键信息无法通过单步方法捕获。为克服此限制，我们提出了CompAct——一种采用主动策略来压缩长文档且不丢失关键信息的新型框架。实验表明，CompAct在多跳问答基准测试中，在性能与压缩率方面均带来显著提升。该框架可作为高性价比的插件模块，与多种现成的检索器或阅读器灵活配合，实现极高的压缩率（47倍）。