Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer.

翻译：自然语言处理（NLP）中的提示调优已成为将大型语言模型适配特定任务的日益流行的方法。然而，这些提示（尤其是连续提示）在不同模型之间的可迁移性仍是一个挑战。在本文中，我们提出了一种零样本连续提示迁移方法，将源提示编码到相对空间中，并搜索对应的目标提示以迁移至目标模型。实验结果证实了我们方法的有效性，表明连续提示中的“任务语义”可以跨多种语言模型进行泛化。此外，我们发现结合来自多个源模型的“任务语义”能进一步增强迁移的泛化能力。