Entities, as important carriers of real-world knowledge, play a key role in many NLP tasks. We focus on incorporating entity knowledge into an encoder-decoder framework for informative text generation. Existing approaches tried to index, retrieve, and read external documents as evidence, but they suffered from a large computational overhead. In this work, we propose an encoder-decoder framework with an entity memory, namely EDMem. The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder parameters. To precisely generate entity names, we design three decoding methods to constrain entity generation by linking entities in the memory. EDMem is a unified framework that can be used on various entity-intensive question answering and generation tasks. Extensive experimental results show that EDMem outperforms both memory-based auto-encoder models and non-memory encoder-decoder models.

翻译：实体作为现实世界知识的重要载体，在许多自然语言处理任务中发挥着关键作用。本文聚焦于将实体知识融入编码器-解码器框架，以实现信息性文本生成。现有方法尝试索引、检索并读取外部文档作为证据，但面临巨大的计算开销。为此，我们提出一种带有实体记忆的编码器-解码器框架，即EDMem。实体知识以潜在表示形式存储于记忆中，该记忆与编码器-解码器参数一同在维基百科上进行预训练。为精确生成实体名称，我们设计了三种解码方法，通过链接记忆中的实体来约束实体生成。EDMem是一种统一框架，可应用于各类实体密集型问答与生成任务。大量实验结果表明，EDMem在性能上优于基于记忆的自编码器模型及无记忆的编码器-解码器模型。