Recent advances in general purpose pre-trained language models have shown great potential in commonsense reasoning. However, current works still perform poorly on standard commonsense reasoning benchmarks including the Com2Sense Dataset. We argue that this is due to a disconnect with current cutting-edge machine learning methods. In this work, we aim to bridge the gap by introducing current ML-based methods to improve general purpose pre-trained language models in the task of commonsense reasoning. Specifically, we experiment with and systematically evaluate methods including knowledge transfer, model ensemble, and introducing an additional pairwise contrastive objective. Our best model outperforms the strongest previous works by ~15\% absolute gains in Pairwise Accuracy and ~8.7\% absolute gains in Standard Accuracy.

翻译：通用预训练语言模型的最新进展在常识推理中展现出巨大潜力。然而，当前模型在包括Com2Sense数据集在内的标准常识推理基准上表现仍不理想。我们认为这源于其与前沿机器学习方法的脱节。本研究旨在通过引入基于当代机器学习的方法，弥合这一差距，以提升通用预训练语言模型在常识推理任务中的表现。具体而言，我们系统实验并评估了知识迁移、模型集成以及引入成对对比目标函数等方法。最优模型在成对准确率上较之前最强基线实现约15%的绝对提升，在标准准确率上实现约8.7%的绝对提升。