The latest developments in Natural Language Processing (NLP) have demonstrated remarkable progress in a code-text retrieval problem. As the Transformer-based models used in this task continue to increase in size, the computational costs and time required for end-to- end fine-tuning become substantial. This poses a significant challenge for adapting and utilizing these models when computational resources are limited. Motivated by these concerns, we propose a fine-tuning frame- work that leverages Parameter-Efficient Fine-Tuning (PEFT) techniques. Moreover, we adopt contrastive learning objectives to improve the quality of bimodal representations learned by transformer models. Additionally, for PEFT methods we provide extensive benchmarking, the lack of which has been highlighted as a crucial problem in the literature. Based on the thorough experimentation with the CodeT5+ model conducted on two datasets, we demonstrate that the proposed fine-tuning framework has the potential to improve code-text retrieval performance by tuning only 0.4% parameters at most.

翻译：自然语言处理领域的最新进展在代码-文本检索问题上取得了显著突破。然而，随着该任务中所使用的基于Transformer的模型规模持续增大，端到端微调所需的计算成本和时间变得相当可观。当计算资源受限时，这为适配和利用这些模型带来了重大挑战。基于上述考量，我们提出了一种利用参数高效微调技术的微调框架。此外，我们采用对比学习目标来提升Transformer模型所学习的双模态表示质量。同时，针对参数高效微调方法，我们提供了全面的基准测试，而相关文献已明确指出缺乏此类基准测试是一个关键问题。通过在两个数据集上对CodeT5+模型进行深入实验，我们证明了所提出的微调框架仅需至多调整0.4%的参数，即具备提升代码-文本检索性能的潜力。