Large Language Models (LLMs) have showcased impressive capabilities in handling straightforward programming tasks. However, their performance tends to falter when confronted with more challenging programming problems. We observe that conventional models often generate solutions as monolithic code blocks, restricting their effectiveness in tackling intricate questions. To overcome this limitation, we present Modular-of-Thought Coder (MoTCoder). We introduce a pioneering framework for MoT instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules. Our investigations reveal that, through the cultivation and utilization of sub-modules, MoTCoder significantly improves both the modularity and correctness of the generated solutions, leading to substantial relative pass@1 improvements of 12.9% on APPS and 9.43% on CodeContests. Our codes are available at https://github.com/dvlab-research/MoTCoder.

翻译：大语言模型（LLMs）在处理简单编程任务时已展现出令人印象深刻的能力。然而，当面对更具挑战性的编程问题时，其性能往往会出现下降。我们观察到，传统模型通常将解决方案生成为单一的整体代码块，这限制了其处理复杂问题的有效性。为克服这一局限，我们提出了模块化思维编程器（MoTCoder）。我们引入了一种开创性的MoT指令调优框架，旨在促进将任务分解为逻辑子任务和子模块。我们的研究表明，通过培养和利用子模块，MoTCoder显著提高了生成解决方案的模块化程度和正确性，从而在APPS和CodeContests基准上分别实现了12.9%和9.43%的相对pass@1显著提升。我们的代码公开于https://github.com/dvlab-research/MoTCoder。