Recently, large language models (LLMs) have achieved significant progress in automated code generation. Despite their strong instruction-following capabilities, these models frequently struggled to align with user intent in coding scenarios. In particular, they were hampered by datasets that lacked diversity and failed to address specialized tasks or edge cases. Furthermore, challenges in supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) led to failures in generating precise, human-intent-aligned code. To tackle these challenges and improve the code generation performance for automated programming systems, we propose Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization (i.e., FALCON). FALCON is structured into two hierarchical levels. From the global level, long-term memory improves code quality by retaining and applying learned knowledge. At the local level, short-term memory allows for the incorporation of immediate feedback from compilers and AI systems. Additionally, we introduce meta-reinforcement learning with feedback rewards to solve the global-local bi-level optimization problem and enhance the model's adaptability across diverse code generation tasks. Extensive experiments demonstrate that our technique achieves state-of-the-art performance, leading other reinforcement learning methods by more than 4.5 percentage points on the MBPP benchmark and 6.1 percentage points on the Humaneval benchmark. The open-sourced code is publicly available at https://github.com/titurte/FALCON.

翻译：近年来，大型语言模型（LLMs）在自动化代码生成方面取得了显著进展。尽管这些模型具备强大的指令跟随能力，但在编码场景中往往难以与用户意图保持一致。特别是，它们受到数据集多样性不足的制约，无法有效处理专业任务或边缘案例。此外，监督微调（SFT）和基于人类反馈的强化学习（RLHF）中存在的挑战，导致模型难以生成精确且符合人类意图的代码。为应对这些挑战并提升自动化编程系统的代码生成性能，本文提出反馈驱动的自适应长短时记忆强化编码优化系统（即FALCON）。FALCON采用双层结构设计：在全局层面，长时记忆通过保留并应用已学知识来提升代码质量；在局部层面，短时记忆支持融入来自编译器和AI系统的即时反馈。此外，我们引入基于反馈奖励的元强化学习方法，以解决全局-局部双层优化问题，并增强模型在多样化代码生成任务中的适应性。大量实验表明，本方法实现了最先进的性能，在MBPP基准测试中领先其他强化学习方法超过4.5个百分点，在Humaneval基准测试中领先6.1个百分点。开源代码已发布于https://github.com/titurte/FALCON。