The utilization of programming language (PL) models, pretrained on large-scale code corpora, as a means of automating software engineering processes has demonstrated considerable potential in streamlining various code generation tasks such as code completion, code translation, and program synthesis. However, current approaches mainly rely on supervised fine-tuning objectives borrowed from text generation, neglecting specific sequence-level features of code, including but not limited to compilability as well as syntactic and functional correctness. To address this limitation, we propose PPOCoder, a new framework for code generation that combines pretrained PL models with Proximal Policy Optimization (PPO) deep reinforcement learning and employs execution feedback as the external source of knowledge into the model optimization. PPOCoder is transferable across different code generation tasks and PLs. Extensive experiments on three code generation tasks demonstrate the effectiveness of our proposed approach compared to SOTA methods, improving the success rate of compilation and functional correctness over different PLs. Our code can be found at https://github.com/reddy-lab-code-research/PPOCoder .

翻译：利用预训练于大规模代码语料库的编程语言模型来自动化软件工程流程的方法，在代码补全、代码翻译和程序合成等各类代码生成任务中展现出巨大潜力。然而，当前方法主要依赖从文本生成迁移而来的监督微调目标，忽视了代码特有的序列级特征，包括但不限于可编译性以及语法和功能正确性。为解决这一局限，我们提出PPOCoder——一种结合预训练编程语言模型与近端策略优化深度强化学习的新框架，并将执行反馈作为外部知识源纳入模型优化过程。PPOCoder可迁移应用于不同代码生成任务和编程语言。在三个代码生成任务上的大量实验表明，与最先进方法相比，我们提出的方法能够有效提升不同编程语言下的编译成功率与功能正确性。我们的代码可在https://github.com/reddy-lab-code-research/PPOCoder 获取。