Fine-tuning pre-trained language models (PLMs), e.g., SciBERT, generally requires large numbers of annotated data to achieve state-of-the-art performance on a range of NLP tasks in the scientific domain. However, obtaining the fine-tune data for scientific NLP task is still challenging and expensive. Inspired by recent advancement in prompt learning, in this paper, we propose the Mix Prompt Tuning (MPT), which is a semi-supervised method to alleviate the dependence on annotated data and improve the performance of multi-granularity academic function recognition tasks with a small number of labeled examples. Specifically, the proposed method provides multi-perspective representations by combining manual prompt templates with automatically learned continuous prompt templates to help the given academic function recognition task take full advantage of knowledge in PLMs. Based on these prompt templates and the fine-tuned PLM, a large number of pseudo labels are assigned to the unlabeled examples. Finally, we fine-tune the PLM using the pseudo training set. We evaluate our method on three academic function recognition tasks of different granularity including the citation function, the abstract sentence function, and the keyword function, with datasets from computer science domain and biomedical domain. Extensive experiments demonstrate the effectiveness of our method and statistically significant improvements against strong baselines. In particular, it achieves an average increase of 5% in Macro-F1 score compared with fine-tuning, and 6% in Macro-F1 score compared with other semi-supervised method under low-resource settings. In addition, MPT is a general method that can be easily applied to other low-resource scientific classification tasks.

翻译：微调预训练语言模型（如SciBERT）通常需要大量标注数据才能在科学领域的多种自然语言处理任务上达到最优性能。然而，为科学自然语言处理任务获取微调数据仍然具有挑战性且成本高昂。受提示学习最新进展的启发，本文提出混合提示调优方法，这是一种半监督方法，旨在减轻对标注数据的依赖，并利用少量标注样本提升多粒度学术功能识别任务的性能。具体而言，该方法通过结合人工设计的提示模板与自动学习的连续提示模板，提供多视角的表示，以帮助给定的学术功能识别任务充分利用预训练语言模型中的知识。基于这些提示模板和微调后的预训练语言模型，我们为大量未标注样本分配伪标签。最后，我们使用伪训练集对预训练语言模型进行微调。我们在计算机科学和生物医学领域的三个不同粒度的学术功能识别任务（包括引用功能、摘要句子功能和关键词功能）上评估了所提方法。大量实验证明了该方法的有效性，相比强基线模型取得了统计上显著的性能提升。特别是在低资源设置下，与直接微调相比，其Macro-F1分数平均提升5%；与其他半监督方法相比，平均提升6%。此外，混合提示调优是一种通用方法，可轻松应用于其他低资源科学分类任务。