The dominant text generation models compose the output by sequentially selecting words from a fixed vocabulary. In this paper, we formulate text generation as progressively copying text segments (e.g., words or phrases) from an existing text collection. We compute the contextualized representations of meaningful text segments and index them using efficient vector search toolkits. The task of text generation is then decomposed into a series of copy-and-paste operations: at each time step, we seek suitable text spans from the text collection rather than selecting from a standalone vocabulary. Experiments on the standard language modeling benchmark (WikiText-103) show that our approach achieves better generation quality according to both automatic and human evaluations. Besides, its inference efficiency is comparable to token-level autoregressive models thanks to the reduction of decoding steps. We also show that our approach allows for effective domain adaptation by simply switching to domain-specific text collection without extra training. Finally, we observe that our approach attains additional performance gains by simply scaling up to larger text collections, again without further training.\footnote{Our source codes are publicly available at \url{https://github.com/gmftbyGMFTBY/Copyisallyouneed}.}

翻译：主导文本生成模型通过从固定词汇表中顺序选择词语来构建输出。在本文中，我们将文本生成形式化为从一个现有文本集合中逐步复制文本片段（例如，单词或短语）。我们计算有意义文本片段的上下文表示，并利用高效向量搜索工具包对其进行索引。然后，文本生成任务被分解为一系列复制粘贴操作：在每个时间步，我们从文本集合中寻找合适的文本跨度，而非从独立词汇表中进行选择。在标准语言建模基准测试（WikiText-103）上的实验表明，根据自动评估和人工评估，我们的方法实现了更好的生成质量。此外，由于解码步骤的减少，其推理效率与单词级自回归模型相当。我们还展示了该方法能够通过简单地切换为领域特定的文本集合进行有效的领域适应，而无需额外训练。最后，我们观察到，仅通过扩展至更大的文本集合（同样无需进一步训练），我们的方法便能获得额外的性能提升。\footnote{我们的源代码已在 \url{https://github.com/gmftbyGMFTBY/Copyisallyouneed} 公开。}