While large language model (LLM) multi-agent systems achieve superior reasoning performance through iterative debate, practical deployment is limited by their high computational cost and error propagation. This paper proposes AgentArk, a novel framework to distill multi-agent dynamics into the weights of a single model, effectively transforming explicit test-time interactions into implicit model capabilities. This equips a single agent with the intelligence of multi-agent systems while remaining computationally efficient. Specifically, we investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios: reasoning-enhanced fine-tuning; trajectory-based augmentation; and process-aware distillation. By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents. They further demonstrate enhanced robustness and generalization across diverse reasoning tasks. We hope this work can shed light on future research on efficient and robust multi-agent development. Our code is at https://github.com/AIFrontierLab/AgentArk.

翻译：尽管大型语言模型多智能体系统通过迭代辩论实现了卓越的推理性能，但其实际部署受限于高昂的计算成本和错误传播。本文提出AgentArk——一种将多智能体动态过程蒸馏至单一模型权重中的新型框架，有效将显式的测试时交互转化为隐式模型能力。这使得单一智能体在保持计算高效性的同时，具备多智能体系统的智能水平。具体而言，我们研究了跨多种模型、任务、规模及场景的三种分层蒸馏策略：推理增强微调、基于轨迹的数据增强、以及过程感知蒸馏。通过将计算负担从推理阶段转移至训练阶段，蒸馏后的模型既保留了单一智能体的效率，又展现出多智能体系统的强推理与自我修正能力。此外，这些模型在多样化推理任务中表现出更强的鲁棒性与泛化性。我们希望这项工作能为高效鲁棒的多智能体系统开发提供启示。我们的代码开源在https://github.com/AIFrontierLab/AgentArk。