Multi-Agent Systems (MAS) built using AI agents fulfill a variety of user intents that may be used to design and build a family of related applications. However, the creation of such MAS currently involves manual composition of the plan, manual selection of appropriate agents, and manual creation of execution graphs. This paper introduces a framework for the automated creation of multi-agent systems which replaces multiple manual steps with an automated framework. The proposed framework consists of software modules and a workflow to orchestrate the requisite task- specific application. The modules include: an LLM-derived planner, a set of tasks described in natural language, a dynamic call graph, an orchestrator for map agents to tasks, and an agent recommender that finds the most suitable agent(s) from local and global agent registries. The agent recommender uses a two-stage information retrieval (IR) system comprising a fast retriever and an LLM-based re-ranker. We implemented a series of experiments exploring the choice of embedders, re- rankers, agent description enrichment, and supervising critique agent. We benchmarked this system end-to-end, evaluating the combination of planning, agent selection, and task completion, with our proposed approach. Our experimental results show that our approach outperforms the state-of-the- art in terms of the recall rate and is more robust and scalable compared to previous approaches. The critique agent holistically reevaluates both agent and tool recommendations against the overall plan. We show that the inclusion of the critique agent further enhances the recall score, proving that the comprehensive review and revision of task-based agent selection is an essential step in building end-to-end multi-agent systems.

翻译：基于AI智能体构建的多智能体系统（MAS）可满足多样化的用户意图，用以设计和开发一系列相关应用。然而，当前此类MAS的创建过程仍依赖于手动组合计划、手动选择合适智能体以及手动构建执行图。本文提出一种自动化创建多智能体系统的框架，将多个手动步骤替换为自动化的框架流程。该框架包含软件模块及工作流两部分，用于编排所需的特定任务应用。其中模块包括：基于大语言模型（LLM）推导的规划器、以自然语言描述的任务集合、动态调用图、任务-智能体映射编排器，以及从本地与全局智能体注册库中筛选最优智能体的推荐器。智能体推荐器采用两阶段信息检索（IR）系统，由快速检索器与基于LLM的重排序器构成。我们设计系列实验，探索了嵌入器选择、重排序器选择、智能体描述增强及监督型评审智能体的作用，并通过端到端系统基准测试，评估了所提方法在规划、智能体选择与任务完成三个维度的综合表现。实验结果表明：本方法在召回率指标上优于现有最先进技术，且比先前方法更具鲁棒性和可扩展性。评审智能体基于整体计划对智能体与工具推荐进行全局重评估，其引入进一步提升了召回率评分——这证明基于任务的智能体选择需经全面审查与修正，是构建端到端多智能体系统的关键步骤。