With the rapid advancement of Large Language Models (LLMs), the demand for robust instruction-following capabilities in code generation tasks has grown significantly. Code generation not only facilitates faster prototyping and automated testing, but also augments developer efficiency through improved maintainability and reusability of code. In this paper, we introduce CodeIF, the first benchmark specifically designed to assess the abilities of LLMs to adhere to task-oriented instructions within diverse code generation scenarios. CodeIF encompasses a broad range of tasks, including function synthesis, error debugging, algorithmic refactoring, and code explanation, thereby providing a comprehensive suite to evaluate model performance across varying complexity levels and programming domains. We conduct extensive experiments with LLMs, analyzing their strengths and limitations in meeting the demands of these tasks. The experimental results offer valuable insights into how well current models align with human instructions, as well as the extent to which they can generate consistent, maintainable, and contextually relevant code. Our findings not only underscore the critical role that instruction-following LLMs can play in modern software development, but also illuminate pathways for future research aimed at enhancing their adaptability, reliability, and overall effectiveness in automated code generation.

翻译：随着大语言模型的快速发展，代码生成任务中对鲁棒指令遵循能力的需求显著增长。代码生成不仅有助于加速原型设计和自动化测试，还能通过提升代码的可维护性和可复用性来增强开发效率。本文提出CodeIF——首个专门用于评估大语言模型在多样化代码生成场景中遵循任务导向指令能力的基准测试。CodeIF涵盖广泛的任务类型，包括函数合成、错误调试、算法重构和代码解释，从而提供了一套能够评估模型在不同复杂度级别和编程领域表现的综合性测试集。我们通过大量实验对大语言模型进行测试，分析其在满足这些任务需求时的优势与局限。实验结果为了解当前模型与人类指令的契合程度，以及其生成一致性、可维护性且符合上下文代码的能力提供了重要参考。我们的发现不仅突显了具备指令遵循能力的大语言模型在现代软件开发中的关键作用，也为未来研究指明了方向，旨在提升其在自动化代码生成中的适应性、可靠性和整体效能。