Large Language Models (LLMs) have showcased impressive capabilities in handling straightforward programming tasks. However, their performance tends to falter when confronted with more challenging programming problems. We observe that conventional models often generate solutions as monolithic code blocks, restricting their effectiveness in tackling intricate questions. To overcome this limitation, we present Modular-of-Thought Coder (MoTCoder). We introduce a pioneering framework for MoT instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules. Our investigations reveal that, through the cultivation and utilization of sub-modules, MoTCoder significantly improves both the modularity and correctness of the generated solutions, leading to substantial relative pass@1 improvements of 12.9% on APPS and 9.43% on CodeContests. Our codes are available at https://github.com/dvlab-research/MoTCoder.

翻译：大语言模型在处理简单编程任务时已展现出令人瞩目的能力。然而，面对更具挑战性的编程问题时，其性能往往会下降。我们观察到，传统模型通常将解决方案生成为整体代码块，这限制了它们在解决复杂问题时的有效性。为克服这一局限，我们提出了模块化思维编码器。我们引入了一种创新的MoT指令微调框架，旨在促进任务分解为逻辑子任务和子模块。研究表明，通过培养和利用子模块，MoTCoder显著提升了生成解决方案的模块化程度与正确性，在APPS和CodeContests数据集上分别带来了12.9%和9.43%的相对通过率提升。我们的代码已开源在https://github.com/dvlab-research/MoTCoder。