Multi-agent systems (MAS) built on Large Language Models (LLMs) are being used to approach complex problems and can surpass single model inference. However, their success hinges on navigating a fundamental cognitive tension: the need to balance broad, divergent exploration of the solution space with a principled, convergent synthesis to the optimal solution. Existing paradigms often struggle to manage this duality, leading to premature consensus, error propagation, and a critical credit assignment problem that fails to distinguish between genuine reasoning and superficially plausible arguments. To resolve this core challenge, we propose the Multi-Agent Exploration-Synthesis framework Through Role Orchestration (Maestro), a principled paradigm for collaboration that structurally decouples these cognitive modes. Maestro uses a collective of parallel Execution Agents for diverse exploration and a specialized Central Agent for convergent, evaluative synthesis. To operationalize this critical synthesis phase, we introduce Conditional Listwise Policy Optimization (CLPO), a reinforcement learning objective that disentangles signals for strategic decisions and tactical rationales. By combining decision-focused policy gradients with a list-wise ranking loss over justifications, CLPO achieves clean credit assignment and stronger comparative supervision. Experiments on mathematical reasoning and general problem-solving benchmarks demonstrate that Maestro, coupled with CLPO, consistently outperforms existing state-of-the-art multi-agent approaches, delivering absolute accuracy gains of 6% on average and up to 10% at best.

翻译：基于大语言模型构建的多智能体系统正被用于解决复杂问题，其性能可超越单一模型推理。然而，其成功关键在于应对一个根本的认知张力：需要在解决方案空间中进行广泛、发散的探索，同时进行有原则的、收敛的综合以达成最优解。现有范式往往难以平衡这种二元性，导致过早达成共识、错误传播以及关键的信用分配问题——无法区分真正的推理与表面看似合理的论证。为解决这一核心挑战，我们提出了通过角色编排实现的多智能体探索-综合框架，即Maestro，这是一种结构上解耦上述认知模式的协作原则性范式。Maestro使用一组并行的执行智能体进行多样化探索，并配备一个专门的中枢智能体进行收敛性、评估性综合。为实现这一关键的综合阶段，我们引入了条件列表式策略优化，这是一种强化学习目标，能够分离战略决策信号与战术推理依据。通过将决策导向的策略梯度与基于论证的列表式排序损失相结合，CLPO实现了清晰的信用分配和更强的对比监督。在数学推理和通用问题求解基准上的实验表明，结合CLPO的Maestro持续优于现有的最先进多智能体方法，平均绝对准确率提升6%，最高可达10%。