Prompting pre-trained language models leads to promising results across natural language processing tasks but is less effective when applied in low-resource domains, due to the domain gap between the pre-training data and the downstream task. In this work, we bridge this gap with a novel and lightweight prompting methodology called SwitchPrompt for the adaptation of language models trained on datasets from the general domain to diverse low-resource domains. Using domain-specific keywords with a trainable gated prompt, SwitchPrompt offers domain-oriented prompting, that is, effective guidance on the target domains for general-domain language models. Our few-shot experiments on three text classification benchmarks demonstrate the efficacy of the general-domain pre-trained language models when used with SwitchPrompt. They often even outperform their domain-specific counterparts trained with baseline state-of-the-art prompting methods by up to 10.7% performance increase in accuracy. This result indicates that SwitchPrompt effectively reduces the need for domain-specific language model pre-training.

翻译：对预训练语言模型进行提示（prompting）可在自然语言处理任务中取得显著成果，然而在低资源域中应用时，由于预训练数据与下游任务之间存在域差距，其效果有所下降。本研究提出了一种新颖且轻量级的提示方法——SwitchPrompt，旨在弥补这一差距，实现从通用域数据集训练的语言模型向多样化低资源域的适应。通过使用领域特定关键词与可训练门控提示，SwitchPrompt提供了面向领域的提示机制，即对通用域语言模型在目标域上施加有效引导。我们在三个文本分类基准上进行的少样本实验表明，采用SwitchPrompt的通用域预训练语言模型具备高效性，其性能往往超越使用基线最优提示方法训练的领域特定模型，准确率提升高达10.7%。这一结果表明，SwitchPrompt有效降低了对领域特定语言模型预训练的需求。