Imagine a developer who can only change their last line of code, how often would they have to start writing a function from scratch before it is correct? Auto-regressive models for code generation from natural language have a similar limitation: they do not easily allow reconsidering earlier tokens generated. We introduce CodeFusion, a pre-trained diffusion code generation model that addresses this limitation by iteratively denoising a complete program conditioned on the encoded natural language. We evaluate CodeFusion on the task of natural language to code generation for Bash, Python, and Microsoft Excel conditional formatting (CF) rules. Experiments show that CodeFusion (75M parameters) performs on par with state-of-the-art auto-regressive systems (350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and top-5 accuracy due to its better balance in diversity versus quality.

翻译：设想一位开发者只能修改其最后一行代码，在编写函数时需要从头开始多少次才能得到正确结果？基于自然语言的代码生成自回归模型存在类似局限：它们难以对早期生成的token进行重新修正。我们提出CodeFusion，一种预训练扩散代码生成模型，通过对编码自然语言条件下的完整程序进行迭代去噪来解决这一局限。我们在Bash、Python和Microsoft Excel条件格式化规则三种自然语言到代码生成任务上评估了CodeFusion。实验表明，CodeFusion（7500万参数）在top-1准确率上与最先进的自回归系统（3.5亿-1750亿参数）表现相当，并在top-3和top-5准确率上超越它们，这得益于其在多样性与质量之间更优的平衡。