Code pre-trained models (CodePTMs) have recently demonstrated a solid capacity to process various software intelligence tasks, e.g., code clone detection, code translation, and code summarization. The current mainstream method that deploys these models to downstream tasks is to fine-tune them on individual tasks, which is generally costly and needs sufficient data for large models. To tackle the issue, in this paper, we present TransCoder, a unified Transferable fine-tuning strategy for Code representation learning. Inspired by human inherent skills of knowledge generalization, TransCoder drives the model to learn better code-related meta-knowledge like human programmers. Specifically, we employ a tunable prefix encoder as the meta-learner to capture cross-task and cross-language transferable knowledge, respectively. Besides, tasks with minor training sample sizes and languages with small corpus can be remarkably benefited from our approach. Extensive experiments conducted on benchmark datasets clearly demonstrate that our method can lead to superior performance on various code-related tasks and encourage mutual reinforcement. We also show that TransCoder is applicable in low-resource scenarios. Our codes are available at https://github.com/QiushiSun/TransCoder.

翻译：代码预训练模型（CodePTMs）近期在处理各类软件智能任务（如代码克隆检测、代码翻译及代码摘要生成）中展现出强大能力。当前将这些模型部署至下游任务的主流方法是在单个任务上进行微调，这种方式通常成本高昂且需要大规模模型具备充足数据。为解决该问题，本文提出TransCoder——一种面向代码表示学习的统一可迁移微调策略。受人类知识泛化固有能力的启发，TransCoder驱动模型像程序员一样学习更优的代码相关元知识。具体而言，我们采用可调前缀编码器作为元学习器，分别捕获跨任务与跨语言的可迁移知识。此外，我们的方法能显著受益于训练样本规模较小的任务及语料稀缺的语言。在基准数据集上的大量实验证明，该方法可在各类代码相关任务中取得优越性能并促进相互增强。我们还展示了TransCoder在低资源场景下的适用性。我们的代码已开源至https://github.com/QiushiSun/TransCoder。