Context: Pre-trained models (PTMs) have demonstrated significant potential in automatic code translation. However, the vulnerability of these models in translation tasks, particularly in terms of syntax, has not been extensively investigated. Objective: To fill this gap, our study aims to propose a novel approach CoTR to assess and improve the syntactic adversarial robustness of PTMs in code translation. Method: CoTR consists of two components: CoTR-A and CoTR-D. CoTR-A generates adversarial examples by transforming programs, while CoTR-D proposes a semantic distance-based sampling data augmentation method and adversarial training method to improve the model's robustness and generalization capabilities. The Pass@1 metric is used by CoTR to assess the performance of PTMs, which is more suitable for code translation tasks and offers a more precise evaluation in real world scenarios. Results: The effectiveness of CoTR is evaluated through experiments on real world Java to Python datasets. The results demonstrate that CoTR-A can significantly reduce the performance of existing PTMs, while CoTR-D effectively improves the robustness of PTMs. Conclusion: Our study identifies the limitations of current PTMs, including large language models, in code translation tasks. It highlights the potential of CoTR as an effective solution to enhance the robustness of PTMs for code translation tasks.

翻译：上下文：预训练模型在自动代码翻译中展现出显著潜力。然而，这些模型在翻译任务中的脆弱性，尤其是在句法层面，尚未得到广泛研究。目标：为填补这一空白，本研究旨在提出一种名为CoTR的新方法，用于评估并提升预训练模型在代码翻译中的句法对抗鲁棒性。方法：CoTR由两个组件构成：CoTR-A和CoTR-D。CoTR-A通过程序变换生成对抗样本，而CoTR-D则提出基于语义距离的采样数据增强方法和对抗训练方法，以提升模型的鲁棒性与泛化能力。CoTR使用Pass@1指标评估预训练模型的性能，该指标更适用于代码翻译任务，并能更精确地反映实际场景中的表现。结果：通过在真实Java到Python数据集上的实验，验证了CoTR的有效性。结果表明，CoTR-A能显著降低现有预训练模型的性能，而CoTR-D则有效提升了模型的鲁棒性。结论：本研究指出了当前预训练模型（包括大型语言模型）在代码翻译任务中的局限性，并凸显了CoTR作为提升预训练模型鲁棒性的有效解决方案的潜力。