While existing code large language models (code LLMs) exhibit impressive capabilities in code generation, their autoregressive sequential generation inherently lacks reversibility. This limitation hinders them from timely correcting previous missing statements during coding as humans do, often leading to error propagation and suboptimal performance. We introduce JumpCoder, a novel modelagnostic framework that enables online modification and non-sequential generation to augment the code LLMs. The key idea behind JumpCoder is to insert new code into the currently generated code when necessary during generation, which is achieved through an auxiliary infilling model that works in tandem with the code LLM. Since identifying the best infill position beforehand is intractable, we adopt an infill-first, judge-later strategy, which experiments with filling at the $k$ most critical positions following the generation of each line, and uses an Abstract Syntax Tree (AST) parser alongside the Generation Model Scoring to effectively judge the validity of each potential infill. Extensive experiments using six state-of-the-art code LLMs across multiple benchmarks consistently indicate significant improvements over all baselines. Notably, JumpCoder assists code LLMs in achieving up to a 3.6% increase in Pass@1 for Python, 6.3% for Java, and 3.7% for C++ in the multilingual HumanEval benchmarks. Our code is public at https://github.com/Keytoyze/JumpCoder.

翻译：现有代码大语言模型（code LLMs）虽在代码生成中展现出卓越能力，但其自回归顺序生成本质缺乏可逆性。这一局限阻碍了模型在编码过程中像人类一样及时修正先前遗漏的语句，常导致错误传播和性能欠佳。我们提出JumpCoder——一种新颖的模型无关框架，通过启用在线修改与非顺序生成来增强代码LLMs。其核心思想是在生成过程中，当需要时向当前生成的代码中插入新代码，这通过一个与代码LLM协同工作的辅助填充模型实现。由于事先确定最佳填充位置不可行，我们采用"先填充后评判"策略：在每次生成一行代码后，尝试在$k$个最关键的代码位置进行填充，并利用抽象语法树解析器结合生成模型评分有效评判每个潜在填充的有效性。使用六个最先进的代码LLMs在多个基准测试上的大量实验一致表明，该方法优于所有基线。值得注意的是，在多语言HumanEval基准测试中，JumpCoder帮助代码LLMs在Python上实现Pass@1提升达3.6%，Java提升6.3%，C++提升3.7%。我们的代码已开源至https://github.com/Keytoyze/JumpCoder。