Prompting pre-trained language models leads to promising results across natural language processing tasks but is less effective when applied in low-resource domains, due to the domain gap between the pre-training data and the downstream task. In this work, we bridge this gap with a novel and lightweight prompting methodology called SwitchPrompt for the adaptation of language models trained on datasets from the general domain to diverse low-resource domains. Using domain-specific keywords with a trainable gated prompt, SwitchPrompt offers domain-oriented prompting, that is, effective guidance on the target domains for general-domain language models. Our few-shot experiments on three text classification benchmarks demonstrate the efficacy of the general-domain pre-trained language models when used with SwitchPrompt. They often even outperform their domain-specific counterparts trained with baseline state-of-the-art prompting methods by up to 10.7% performance increase in accuracy. This result indicates that SwitchPrompt effectively reduces the need for domain-specific language model pre-training.

翻译：由于培训前数据与下游任务之间存在领域差距,在低资源领域应用预先培训语言模型时效果不那么好。在这项工作中,我们用一种叫作SwitchPrompt的新颖和轻量级的促进方法来弥补这一差距,该方法被称为SwitchPrompt,用于将受过一般领域数据集培训的语文模型改造成不同的低资源领域。使用具有可训练的门式提示的域名关键词,SwitchPrompt提供面向域的提示,即对通用语言模型的目标领域提供有效的指导。我们在三种文本分类基准上进行的微小实验展示了通用预培训语言模型在使用SwitchPrompt时的功效。它们甚至往往超越了经过基本状态技术提示方法培训的域名对应人员,提高准确性,达到10.7%。这一结果表明,SwitchPrompt实际上减少了对特定语言模型培训前需要的域名语言模型。