Unsupervised commonsense reasoning (UCR) is becoming increasingly popular as the construction of commonsense reasoning datasets is expensive, and they are inevitably limited in their scope. A popular approach to UCR is to fine-tune language models with external knowledge (e.g., knowledge graphs), but this usually requires a large number of training examples. In this paper, we propose to transform the downstream multiple choice question answering task into a simpler binary classification task by ranking all candidate answers according to their reasonableness. To this end, for training the model, we convert the knowledge graph triples into reasonable and unreasonable texts. Extensive experimental results show the effectiveness of our approach on various multiple choice question answering benchmarks. Furthermore, compared with existing UCR approaches using KGs, ours is less data hungry. Our code is available at https://github.com/probe2/BUCA.
翻译:摘要:无监督常识推理(UCR)正日益受到关注,因为构建常识推理数据集成本高昂,且其覆盖范围不可避免地存在局限性。一种流行的UCR方法是利用外部知识(如知识图谱)对语言模型进行微调,但这通常需要大量训练样本。本文提出将下游的多项选择问答任务转化为更简单的二分类任务,通过对所有候选答案的合理性进行排序来实现。为此,在模型训练过程中,我们将知识图谱三元组转换为合理与不合理的文本。大量实验结果表明,我们的方法在多种多项选择问答基准测试中表现有效。此外,与现有使用知识图谱的UCR方法相比,我们的方法对数据的需求更低。我们的代码已开源,地址为https://github.com/probe2/BUCA。