Recent advancements in self-attention neural network architectures have raised the bar for open-ended text generation. Yet, while current methods are capable of producing a coherent text which is several hundred words long, attaining control over the content that is being generated -- as well as evaluating it -- are still open questions. We propose a controlled generation task which is based on expanding a sequence of facts, expressed in natural language, into a longer narrative. We introduce human-based evaluation metrics for this task, as well as a method for deriving a large training dataset. We evaluate three methods on this task, based on fine-tuning pre-trained models. We show that while auto-regressive, unidirectional Language Models such as GPT2 produce better fluency, they struggle to adhere to the requested facts. We propose a plan-and-cloze model (using fine-tuned XLNet) which produces competitive fluency while adhering to the requested content.

翻译：最近自留神经网络结构的进展提高了不限名额文本生成的屏障。然而,虽然目前的方法能够产生一个长达数百字的一致文本,但控制正在产生的内容 -- -- 以及评估它 -- -- 仍然是尚未解决的问题。我们提议了一个基于将自然语言表达的一系列事实扩展为更长期的叙述的受控的一代任务。我们为此任务引入了基于人的评价指标,以及生成大型培训数据集的方法。我们根据经过预先培训的模型,对这项任务的三种方法进行了评估。我们表明,在诸如GPT2等自动递减、单向语言模型产生更好的流利的同时,它们努力遵守所要求的事实。我们提议了一个计划和循环模型(使用经过微调的 XLNet ), 在遵守要求的内容的同时产生有竞争力的流利。