Machine reading comprehension (MRC) is an important area of conversation agents and draws a lot of attention. However, there is a notable limitation to current MRC benchmarks: The labeled answers are mostly either spans extracted from the target corpus or the choices of the given candidates, ignoring the natural aspect of high-quality responses. As a result, MRC models trained on these datasets can not generate human-like responses in real QA scenarios. To this end, we construct a new dataset called Penguin to promote the research of MRC, providing a training and test bed for natural response generation to real scenarios. Concretely, Penguin consists of 200k training data with high-quality fluent, and well-informed responses. Penguin is the first benchmark towards natural response generation in Chinese MRC on a relatively large scale. To address the challenges in Penguin, we develop two strong baselines: end-to-end and two-stage frameworks. Following that, we further design Prompt-BART: fine-tuning the pre-trained generative language models with a mixture of prefix prompts in Penguin. Extensive experiments validated the effectiveness of this design.

翻译：机器阅读理解（MRC）是对话智能体的重要研究方向，并吸引了大量关注。然而，当前MRC基准测试存在一个显著局限：标注答案多是从目标语料中抽取的片段或给定候选项的选择，忽视了高质量回复的自然特性。因此，基于这些数据集训练的MRC模型无法在真实问答场景中生成类人回复。为解决这一问题，我们构建了名为Penguin的新数据集，旨在促进MRC研究，为真实场景下的自然回复生成提供训练和测试平台。具体而言，Penguin包含20万条高质量、流畅且信息丰富的训练数据，是首个面向中文MRC中较大规模自然回复生成的基准数据集。为应对Penguin中的挑战，我们开发了两种强基线方法：端到端框架和两阶段框架。在此基础上，我们进一步设计了Prompt-BART：在Penguin上通过混合前缀提示对预训练生成式语言模型进行微调。大量实验验证了该设计的有效性。