Code generation aims to automatically generate source code from high-level task specifications, which can significantly increase productivity of software engineering. Recently, approaches based on large language models (LLMs) have shown remarkable code generation abilities on simple tasks. However, generate code for more complex tasks, such as competition-level problems, remains challenging. In this paper, we introduce Brainstorm framework for code generation. It leverages a brainstorming step that generates and selects diverse thoughts on the problem to facilitate algorithmic reasoning, where the thoughts are possible blueprint of solving the problem. We demonstrate that Brainstorm significantly enhances the ability of LLMs to solve competition-level programming problems, resulting in a more than 50% increase in the pass@$k$ metrics for ChatGPT on the CodeContests benchmark, achieving state-of-the-art performance. Furthermore, our experiments conducted on LeetCode contests show that our framework boosts the ability of ChatGPT to a level comparable to that of human programmers.

翻译：代码生成旨在根据高层任务规范自动生成源代码，这能够显著提升软件工程的生产力。近期，基于大型语言模型（LLMs）的方法在简单任务上展现出了卓越的代码生成能力。然而，为更复杂的任务（如竞赛级别问题）生成代码仍具挑战。本文中，我们提出了用于代码生成的Brainstorm框架。该框架利用一个头脑风暴步骤，对问题生成并筛选多样化思路以促进算法推理，其中这些思路是解决问题的潜在蓝图。我们证明，Brainstorm显著增强了LLMs解决竞赛级编程问题的能力，使ChatGPT在CodeContests基准上的pass@$k$指标提升超过50%，达到当前最优性能。此外，我们在LeetCode竞赛上进行的实验表明，我们的框架将ChatGPT的能力提升至与人类程序员相当的水平。