Retrieval augmentation is a powerful but expensive method to make language models more knowledgeable about the world. Memory-based methods like LUMEN pre-compute token representations for retrieved passages to drastically speed up inference. However, memory also leads to much greater storage requirements from storing pre-computed representations. We propose MEMORY-VQ, a new method to reduce storage requirements of memory-augmented models without sacrificing performance. Our method uses a vector quantization variational autoencoder (VQ-VAE) to compress token representations. We apply MEMORY-VQ to the LUMEN model to obtain LUMEN-VQ, a memory model that achieves a 16x compression rate with comparable performance on the KILT benchmark. LUMEN-VQ enables practical retrieval augmentation even for extremely large retrieval corpora.

翻译：检索增强是一种强大但成本高昂的方法，旨在使语言模型更深入地理解世界。基于记忆的方法（如LUMEN）通过预先计算检索段落的令牌表示，显著加速推理过程。然而，记忆机制也因存储预计算表示而大幅增加存储需求。我们提出MEMORY-VQ——一种在不牺牲性能的前提下降低记忆增强模型存储需求的新方法。该方法采用向量量化变分自编码器（VQ-VAE）压缩令牌表示。我们将MEMORY-VQ应用于LUMEN模型，得到LUMEN-VQ——一种在KILT基准测试中实现16倍压缩率且性能相当的记忆模型。LUMEN-VQ使得即使面对极其庞大的检索语料库，也能实现实用的检索增强。