Large Language Models (LLMs) have showcased impressive capabilities in handling straightforward programming tasks. However, their performance tends to falter when confronted with more challenging programming problems. We observe that conventional models often generate solutions as monolithic code blocks, restricting their effectiveness in tackling intricate questions. To overcome this limitation, we present Modular-of-Thought Coder (MoTCoder). We introduce a pioneering framework for MoT instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules. Our investigations reveal that, through the cultivation and utilization of sub-modules, MoTCoder significantly improves both the modularity and correctness of the generated solutions, leading to substantial relative pass@1 improvements of 12.9% on APPS and 9.43% on CodeContests. Our codes are available at https://github.com/dvlab-research/MoTCoder.

翻译：大型语言模型在处理简单编程任务时已展现出令人印象深刻的能力。然而，当面对更具挑战性的编程问题时，其性能往往有所下降。我们观察到，传统模型通常以整体代码块形式生成解决方案，这限制了它们在解决复杂问题时的有效性。为克服这一局限，我们提出了模块化思考编程器（MoTCoder）。我们引入了一个创新的MoT指令微调框架，旨在促进任务分解为逻辑子任务和子模块。我们的研究表明，通过培养和利用子模块，MoTCoder显著提升了生成解决方案的模块化程度和正确性，在APPS和CodeContests上分别实现了12.9%和9.43%的显著相对pass@1提升。我们的代码已在https://github.com/dvlab-research/MoTCoder开源。