Prompt quality plays a central role in controlling the behavior, reliability, and reasoning performance of large language models (LLMs), particularly for smaller open-source instruction-tuned models that depend heavily on explicit structure. While recent work has explored automatic prompt optimization using textual gradients and self-refinement, most existing methods treat prompts as monolithic blocks of text, making it difficult to localize errors, preserve critical instructions, or prevent uncontrolled prompt growth. We introduce Modular Prompt Optimization (MPO), a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections, including system role, context, task description, constraints, and output format. MPO applies section-local textual gradients, generated by a critic language model, to refine each section independently while keeping the overall prompt schema fixed. Section updates are consolidated through de-duplication to reduce redundancy and interference between components, yielding an interpretable and robust optimization process. We evaluate MPO on two reasoning benchmarks, ARC-Challenge and MMLU, using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models. Across both benchmarks and models, MPO consistently outperforms an untuned structured prompt and the TextGrad baseline, achieving substantial accuracy gains without modifying model parameters or altering prompt structure. These results demonstrate that maintaining a fixed prompt schema while applying localized, section-wise optimization is an effective and practical approach for improving reasoning performance in small open-source LMs.

翻译：提示质量在控制大型语言模型（LLM）的行为、可靠性和推理性能方面起着核心作用，特别是对于高度依赖显式结构的较小开源指令微调模型。尽管近期研究探索了使用文本梯度和自优化的自动提示优化方法，但现有方法大多将提示视为单一文本块，难以定位错误、保留关键指令或防止提示的无控制增长。本文提出模块化提示优化（MPO），一种基于模式的提示优化框架，将提示视为由固定语义段落组成的结构化对象，包括系统角色、上下文、任务描述、约束条件和输出格式。MPO通过批评语言模型生成的段落局部文本梯度，独立优化每个段落，同时保持整体提示模式固定。段落更新通过去重机制进行整合，以减少组件间的冗余和干扰，从而形成可解释且稳健的优化过程。我们在两个推理基准测试（ARC-Challenge和MMLU）上评估MPO，使用LLaMA-3 8B-Instruct和Mistral-7B-Instruct作为求解模型。在所有基准测试和模型中，MPO始终优于未优化的结构化提示和TextGrad基线，在不修改模型参数或改变提示结构的情况下实现了显著的准确率提升。这些结果表明，在保持固定提示模式的同时应用局部化、分段式优化，是提升小型开源语言模型推理性能的有效且实用的方法。