Fine-tuning pre-trained language models (PLMs), e.g., SciBERT, generally requires large numbers of annotated data to achieve state-of-the-art performance on a range of NLP tasks in the scientific domain. However, obtaining the fine-tune data for scientific NLP task is still challenging and expensive. Inspired by recent advancement in prompt learning, in this paper, we propose the Mix Prompt Tuning (MPT), which is a semi-supervised method to alleviate the dependence on annotated data and improve the performance of multi-granularity academic function recognition tasks with a small number of labeled examples. Specifically, the proposed method provides multi-perspective representations by combining manual prompt templates with automatically learned continuous prompt templates to help the given academic function recognition task take full advantage of knowledge in PLMs. Based on these prompt templates and the fine-tuned PLM, a large number of pseudo labels are assigned to the unlabeled examples. Finally, we fine-tune the PLM using the pseudo training set. We evaluate our method on three academic function recognition tasks of different granularity including the citation function, the abstract sentence function, and the keyword function, with datasets from computer science domain and biomedical domain. Extensive experiments demonstrate the effectiveness of our method and statistically significant improvements against strong baselines. In particular, it achieves an average increase of 5% in Macro-F1 score compared with fine-tuning, and 6% in Macro-F1 score compared with other semi-supervised method under low-resource settings. In addition, MPT is a general method that can be easily applied to other low-resource scientific classification tasks.

翻译：微调预训练语言模型（PLMs），例如SciBERT，通常需要大量标注数据才能在科学领域的各种NLP任务上达到最先进性能。然而，获取科学NLP任务的微调数据仍然具有挑战性且成本高昂。受提示学习最新进展的启发，本文提出了混合提示微调方法（MPT），这是一种半监督方法，旨在减少对标注数据的依赖，并在少量标注样本下提升多粒度学术功能识别任务的性能。具体来说，该方法通过将人工提示模板与自动学习的连续提示模板相结合，提供多视角表示，帮助给定的学术功能识别任务充分利用PLMs中的知识。基于这些提示模板和微调后的PLM，大量伪标签被分配给未标注样本。最后，我们使用伪训练集微调PLM。我们在计算机科学领域和生物医学领域的数据集上，对三种不同粒度的学术功能识别任务（包括引文功能、摘要句子功能和关键词功能）进行了评估。大量实验证明了我们方法的有效性，并在统计上显著优于强基线。特别是在低资源设置下，与微调相比，该方法在Macro-F1得分上平均提升5%，与其他半监督方法相比提升6%。此外，MPT是一种通用方法，可轻松应用于其他低资源的科学分类任务。