There have been many recent investigations into prompt-based training of transformer language models for new text genres in low-resource settings. The prompt-based training approach has been found to be effective in generalizing pre-trained or fine-tuned models for transfer to resource-scarce settings. This work, for the first time, reports results on adopting prompt-based training of transformers for \textit{scholarly knowledge graph object prediction}. The work is unique in the following two main aspects. 1) It deviates from the other works proposing entity and relation extraction pipelines for predicting objects of a scholarly knowledge graph. 2) While other works have tested the method on text genera relatively close to the general knowledge domain, we test the method for a significantly different domain, i.e. scholarly knowledge, in turn testing the linguistic, probabilistic, and factual generalizability of these large-scale transformer models. We find that (i) per expectations, transformer models when tested out-of-the-box underperform on a new domain of data, (ii) prompt-based training of the models achieve performance boosts of up to 40\% in a relaxed evaluation setting, and (iii) testing the models on a starkly different domain even with a clever training objective in a low resource setting makes evident the domain knowledge capture gap offering an empirically-verified incentive for investing more attention and resources to the scholarly domain in the context of transformer models.

翻译：近年来，针对低资源环境下新型文本体裁的Transformer语言模型提示训练进行了大量研究。提示训练方法已被证明能有效泛化预训练或微调模型，使其适应资源匮乏场景。本工作首次报告了将提示训练应用于Transformer进行《学术知识图谱目标预测》的结果。该工作具有以下两个独特方面：1）它偏离了其他采用实体与关系抽取流水线预测学术知识图谱目标的研究；2）其他方法已在相对接近通用知识领域的文本体裁上测试，而我们在显著不同的领域（即学术知识）上测试，从而检验这些大规模Transformer模型在语言学、概率及事实层面的泛化能力。我们发现：（i）如预期所示，Transformer模型在直接部署于新数据领域时性能下降；（ii）在宽松评估设置下，模型的提示训练可带来高达40%的性能提升；（iii）即使在低资源环境下采用巧妙训练目标，于截然不同领域测试模型仍暴露出领域知识捕获差距，这为在Transformer模型背景下向学术领域投入更多关注和资源提供了经验验证的动机。