A Retrieval-Augmented Language Model (RALM) augments a generative language model by retrieving context-specific knowledge from an external database. This strategy facilitates impressive text generation quality even with smaller models, thus reducing orders of magnitude of computational demands. However, RALMs introduce unique system design challenges due to (a) the diverse workload characteristics between LM inference and retrieval and (b) the various system requirements and bottlenecks for different RALM configurations such as model sizes, database sizes, and retrieval frequencies. We propose Chameleon, a heterogeneous accelerator system that integrates both LM and retrieval accelerators in a disaggregated architecture. The heterogeneity ensures efficient acceleration of both LM inference and retrieval, while the accelerator disaggregation enables the system to independently scale both types of accelerators to fulfill diverse RALM requirements. Our Chameleon prototype implements retrieval accelerators on FPGAs and assigns LM inference to GPUs, with a CPU server orchestrating these accelerators over the network. Compared to CPU-based and CPU-GPU vector search systems, Chameleon achieves up to 23.72x speedup and 26.2x energy efficiency. Evaluated on various RALMs, Chameleon exhibits up to 2.16x reduction in latency and 3.18x speedup in throughput compared to the hybrid CPU-GPU architecture. These promising results pave the way for bringing accelerator heterogeneity and disaggregation into future RALM systems.

翻译：检索增强语言模型通过从外部数据库检索上下文相关知识来增强生成式语言模型。该策略即使使用较小模型也能实现卓越的文本生成质量，从而将计算需求降低数个数量级。然而，由于（a）语言模型推理与检索之间差异化的负载特征，以及（b）不同RALM配置（如模型规模、数据库规模、检索频率）对系统需求与性能瓶颈的差异性，RALM引入了独特的系统设计挑战。我们提出Chameleon——一种异构加速器系统，通过解耦架构集成语言模型加速器与检索加速器。异构性确保了语言模型推理与检索的高效加速，而加速器解耦使系统能够独立扩展两类加速器以满足多样化的RALM需求。Chameleon原型在FPGA上实现检索加速器，将语言模型推理分配至GPU，并通过CPU服务器在网络中协调这些加速器。相较于基于CPU和CPU-GPU的向量搜索系统，Chameleon实现了最高23.72倍的加速比与26.2倍的能效提升。在不同RALM上的评估表明，相较于混合CPU-GPU架构，Chameleon的延迟降低达2.16倍，吞吐量加速比达3.18倍。这些成果为在未来的RALM系统中引入加速器异构性与解耦特性奠定了基础。