In recent years, instruction fine-tuning (IFT) on large language models (LLMs) has garnered considerable attention to enhance model performance on unseen tasks. Attempts have been made on automatic construction and effective selection for IFT data. However, we posit that previous methods have not fully harnessed the potential of LLMs for enhancing data quality. The responses within IFT data could be further enhanced by leveraging the capabilities of LLMs themselves. In this paper, we propose CoEvol, an LLM-based multi-agent cooperation framework for the improvement of responses to instructions. To effectively refine the responses, we develop an iterative framework following a debate-advise-edit-judge paradigm. A two-stage multi-agent debate strategy is further devised to ensure the diversity and reliability of editing suggestions within the framework. Empirically, models equipped with CoEvol outperform competitive baselines evaluated by MT-Bench and AlpacaEval, demonstrating its effectiveness in enhancing instruction-following capabilities for LLMs.

翻译：近年来，基于大型语言模型（LLM）的指令微调（IFT）因其能提升模型在未见任务上的性能而受到广泛关注。已有研究尝试对IFT数据进行自动构建与有效筛选。然而，我们认为先前方法尚未充分利用LLM提升数据质量的潜力。通过利用LLM自身能力，IFT数据中的响应可得到进一步优化。本文提出CoEvol，一种基于LLM的多智能体协同框架，旨在改进对指令的响应。为有效优化响应，我们构建了一个遵循“辩论-建议-编辑-评判”范式的迭代框架。进一步设计了两阶段多智能体辩论策略，以确保框架内编辑建议的多样性与可靠性。实证结果表明，采用CoEvol的模型在MT-Bench和AlpacaEval评估中均优于现有基线，证明了其在增强LLM指令遵循能力方面的有效性。