Multilingual fine-tuning (of a multilingual Pre-trained Language Model) has shown to improve performance of downstream tasks. However, it was observed that different programming languages may have different structural properties, and thus the learning or fine-tuning of a model may be sub-optimal or even degrade the intended performance by using a multilingual dataset. In this study, we proposed a new modular component architecture, AdvFusion, that leverages the different aspects of programming languages for a target popular low-resource programming language, Ruby. Our result shows that AdvFusion can extract useful features from different programming languages efficiently, and it outperforms the existing state-of-the-art multilingual fine-tuning by 12% on the Code Summarization task.

翻译：多语言微调（对多语言预训练语言模型进行微调）已被证明能提升下游任务性能。然而，有研究发现不同编程语言具有不同的结构特性，因此使用多语言数据集进行模型学习或微调可能无法达到最优效果，甚至会降低预期性能。本研究提出了一种新型模块化组件架构AdvFusion，该架构能够充分利用不同编程语言的多方面特征，针对目标低资源编程语言Ruby进行优化。实验结果表明，AdvFusion能够高效地从不同编程语言中提取有用特征，在代码摘要任务上比现有最先进的多语言微调方法性能提升12%。