Recently, pretrained language models (e.g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability. However, their performance on commonsense tasks is still far from that of humans. As a preliminary attempt, we propose a simple yet effective method to teach pretrained models with commonsense reasoning by leveraging the structured knowledge in ConceptNet, the largest commonsense knowledge base (KB). Specifically, the structured knowledge in KB allows us to construct various logical forms, and then generate multiple-choice questions requiring commonsense logical reasoning. Experimental results demonstrate that, when refined on these training examples, the pretrained models consistently improve their performance on tasks that require commonsense reasoning, especially in the few-shot learning setting. Besides, we also perform analysis to understand which logical relations are more relevant to commonsense reasoning.

翻译：近年来，预训练语言模型（如BERT）在诸多下游自然语言理解任务上取得了巨大成功，并展现出一定程度的常识推理能力。然而，它们在常识任务上的表现仍远不及人类。作为一项初步尝试，我们提出一种简单而有效的方法，通过利用最大常识知识库ConceptNet中的结构化知识，为预训练模型注入常识推理能力。具体而言，知识库中的结构化知识使我们能够构建多种逻辑形式，进而生成需要常识逻辑推理的多选题。实验结果表明，通过在这些训练样本上进行微调，预训练模型在需要常识推理的任务上性能持续提升，尤其在少样本学习场景中效果显著。此外，我们还进行了分析以理解哪些逻辑关系与常识推理更为相关。