Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing (NLP) tasks such as text classification, question answering and so on. However, existing methods that separately or asynchronously train the retriever and downstream model mainly due to the non-differentiability between the two parts, usually lead to degraded performance compared to end-to-end joint training. In this paper, we propose Differentiable Retrieval Augmentation via Generative lANguage modeling(Dragan), to address this problem by a novel differentiable reformulation. We demonstrate the effectiveness of our proposed method on a challenging NLP task in e-commerce search, namely query intent classification. Both the experimental results and ablation study show that the proposed method significantly and reasonably improves the state-of-the-art baselines on both offline evaluation and online A/B test.

翻译：检索增强通过知识检索器和外部语料库提升下游模型性能，而非单纯增加模型参数数量，该方法已成功应用于文本分类、问答等多项自然语言处理任务。然而，现有方法因检索器与下游模型之间存在的不可微性问题，通常采用分离或异步训练方式，导致其性能低于端到端联合训练。本文提出基于生成语言模型的可微分检索增强方法(Dragan)，通过新颖的可微分重构方案解决该问题。我们在电商搜索领域的挑战性自然语言处理任务——查询意图分类中验证了所提方法的有效性。实验结果表明，在离线评估和在线A/B测试中，该方法均能显著且合理地提升现有最优基线模型的性能。消融实验进一步证实了各模块的有效性。