Large Language Models (LLMs) have showcased impressive capabilities in handling straightforward programming tasks. However, their performance tends to falter when confronted with more challenging programming problems. We observe that conventional models often generate solutions as monolithic code blocks, restricting their effectiveness in tackling intricate questions. To overcome this limitation, we present Module-of-Thought Coder (MoTCoder). We introduce a framework for MoT instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules. Our investigations reveal that, through the cultivation and utilization of sub-modules, MoTCoder significantly improves both the modularity and correctness of the generated solutions, leading to substantial pass@1 improvements of 5.9% on APPS and 5.8% on CodeContests. MoTCoder also achieved significant improvements in self-correction capabilities, surpassing the current SOTA by 3.3%. Additionally, we provide an analysis of between problem complexity and optimal module decomposition and evaluate the maintainability index, confirming that the code generated by MoTCoder is easier to understand and modify, which can be beneficial for long-term code maintenance and evolution. Our codes are available at https://github.com/dvlab-research/MoTCoder.

翻译：大语言模型在处理简单编程任务方面已展现出令人印象深刻的能力。然而，当面对更具挑战性的编程问题时，其性能往往有所下降。我们观察到，传统模型通常将解决方案生成为单一的整体代码块，这限制了其处理复杂问题的有效性。为克服这一局限，我们提出了模块化思维编码器。我们引入了一个用于模块化思维指令微调的框架，旨在促进将任务分解为逻辑子任务和子模块。我们的研究表明，通过培养和利用子模块，MoTCoder显著提高了生成解决方案的模块化程度和正确性，在APPS数据集上实现了5.9%的pass@1显著提升，在CodeContests数据集上提升了5.8%。MoTCoder在自我纠错能力方面也取得了显著改进，超越了当前最优方法3.3%。此外，我们分析了问题复杂度与最优模块分解之间的关系，并评估了可维护性指数，证实MoTCoder生成的代码更易于理解和修改，这对长期代码维护和演进具有积极意义。我们的代码公开于https://github.com/dvlab-research/MoTCoder。