Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer.

翻译：自然语言处理（NLP）中的提示调优已成为一种日益流行的方法，用于使大型语言模型适应特定任务。然而，这些提示（尤其是连续提示）在不同模型间的可迁移性仍然是一个挑战。在本工作中，我们提出了一种零样本连续提示迁移方法，其中源提示被编码到相对空间中，并通过搜索对应的目标提示以迁移到目标模型。实验结果证实了我们方法的有效性，表明连续提示中的“任务语义”可以在不同语言模型间实现泛化。此外，我们发现结合多个源模型的“任务语义”可以进一步提升迁移的泛化能力。