The translation of natural language to formal constraint models requires expertise in the problem domain and modeling frameworks. To explore the effectiveness of agentic workflows, we propose CP-Agent, a Python coding agent that uses the ReAct framework with a persistent IPython kernel. We provide the relevant domain knowledge as a project prompt of under 50 lines. The algorithm works by iteratively executing code, observing the solver's feedback, and refining constraint models based on execution results. We evaluate CP-Agent on 101 constraint programming problems from CP-Bench. We made minor changes to the benchmark to address systematic ambiguities in the problem specifications and errors in the ground-truth models. On the clarified benchmark, CP-Agent achieves perfect accuracy on all 101 problems. Our experiments show that minimal guidance outperforms detailed procedural scaffolding. Our experiments also show that explicit task management tools can have both positive and negative effects on focused modeling tasks.
翻译:将自然语言转换为形式化约束模型需要问题领域和建模框架方面的专业知识。为探索智能体工作流的有效性,我们提出CP-Agent——一个采用ReAct框架并基于持久化IPython内核的Python编码智能体。我们将相关领域知识以不超过50行的项目提示形式提供。该算法通过迭代执行代码、观察求解器反馈,并根据执行结果优化约束模型来工作。我们在CP-Bench的101个约束编程问题上对CP-Agent进行评估。我们对基准测试进行了微调,以解决问题描述中系统性的歧义以及真实模型中的错误。在澄清后的基准测试中,CP-Agent在所有101个问题上均实现了完全准确。实验表明,最小化指导优于详细的过程化框架。我们的实验还表明,显式任务管理工具对聚焦建模任务可能同时产生积极与消极影响。