We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Existing question generation systems widely differentiate questions by $\textit{literal}$ information such as question words and answer types to generate semantically relevant questions for a given context. However, they rarely consider the $\textit{comprehension}$ nature of questions, i.e. the different comprehension capabilities embodied by different questions. In comparison, our $\texttt{SkillQG}$ is able to tailor a fine-grained assessment and improvement to the capabilities of question answering models built on it. Specifically, we first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate $\texttt{SkillQG}$ as a skill-conditioned question generator. Furthermore, to improve the controllability of generation, we augment the input text with question focus and skill-specific knowledge, which are constructed by iteratively prompting the pre-trained language models. Empirical results demonstrate that $\texttt{SkillQG}$ outperforms baselines in terms of quality, relevance, and skill-controllability while showing a promising performance boost in downstream question answering task.

翻译：我们提出$\textbf{$\texttt{SkillQG}$}$：一个具备可控理解类型的题目生成框架，用于评估和改进机器阅读理解模型。现有题目生成系统主要依据问题词和答案类型等$\textit{字面}$信息来区分类别，从而为给定上下文生成语义相关的问题。然而，这些系统极少考虑问题的$\textit{理解}$本质，即不同问题所体现的不同理解能力。相比之下，我们的$\texttt{SkillQG}$能够针对基于该框架构建的问答模型的能力进行细粒度评估与改进。具体而言，我们首先基于分层技能模式定义问题的理解类型，然后将$\texttt{SkillQG}$构建为技能条件化的题目生成器。为提升生成结果的可控性，我们通过迭代提示预训练语言模型来构建问题焦点与技能特定知识，并将其融入输入文本。实验结果表明，$\texttt{SkillQG}$在质量、相关性和技能可控性方面均优于基线模型，同时在下游问答任务中展现出显著的性能提升。