Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing (NLP) tasks such as text classification, question answering and so on. However, existing methods that separately or asynchronously train the retriever and downstream model mainly due to the non-differentiability between the two parts, usually lead to degraded performance compared to end-to-end joint training.

翻译：检索增强技术通过知识检索器与外部语料库增强下游模型，而非单纯增加模型参数量，已成功应用于文本分类、问答等多项自然语言处理任务。然而现有方法中，由于检索器与下游模型之间存在不可微性，两者通常采用分离式或异步训练方式，这往往导致其性能不及端到端联合训练方法。