Recent research explores optimization using large language models (LLMs) by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer. However, these approaches exhibit inherent limitations, including low operational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. Specifically, we establish a comprehensive instruction set containing well-described problem prompts and effective optimization codes. We then develop a novel two-phase learning strategy that incorporates a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance the convergence behavior during model fine-tuning. The experiment results demonstrate that a CodeGen (350M) model fine-tuned by our LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and the other competitors across both synthetic and realistic problem sets. The fine-tuned model and the usage instructions are available at https://anonymous.4open.science/r/LLaMoCo-722A.

翻译：近期研究探索了利用大语言模型（LLMs）进行优化，其方式包括迭代式地向LLMs寻求下一步解决方案，或直接提示LLMs生成优化器。然而，这些方法存在固有局限性，包括运行效率低、对提示设计高度敏感以及缺乏领域特定知识。我们提出LLaMoCo——首个旨在通过代码到代码方式适配LLMs解决优化问题的指令微调框架。具体而言，我们构建了一个包含详尽问题描述提示与高效优化代码的综合性指令集，并开发了一种新颖的两阶段学习策略：在指令微调阶段前引入基于对比学习的预热过程，以增强模型微调期间的收敛性能。实验结果表明，经LLaMoCo微调的CodeGen（350M）模型在合成问题集与真实问题集上均展现出优于GPT-4 Turbo及其他竞争方法的优化性能。微调后的模型及使用说明已开源至https://anonymous.4open.science/r/LLaMoCo-722A。