Multimodal agents have achieved notable progress on complex reasoning tasks through tool use, yet remain limited by two issues: statically predefined tool inventories fail to generalize to unseen scenarios, and indiscriminate tool invocation incurs redundant cost and noise-induced errors. We propose MetaForge, a multimodal agent framework that learns when to invoke tools and how to evolve its toolset on demand. MetaForge factorizes agentic behavior into four coupled stages: Decide (judging whether tool use is warranted), Retrieve (selecting suitable tools), Adapt (grounding tool parameters in task context), and Forge (synthesizing new skills online and recycling them into the tool library for reuse), forming a closed judge-retrieve-adapt-forge-recycle loop. A unified orchestration policy enables the agent to choose among answering directly, reusing existing tools, or forging new ones. We jointly optimize invocation necessity, retrieval accuracy, execution effectiveness, and forged-skill reusability via reinforcement learning, with an explicit invocation-cost penalty discouraging redundant calls. Across 12 benchmarks, MetaForge consistently surpasses 16 baselines in accuracy, efficiency, and generalization, validating a paradigm shift from static tool inventories to on-demand self-evolution.
翻译:多模态智能体通过工具使用在复杂推理任务上取得了显著进展,但仍受限于两个问题:静态预定义的工具库无法泛化至未见场景,且无差别化的工具调用会引发冗余开销与噪声诱导错误。我们提出MetaForge——一种学习何时调用工具并按需进化工具库的多模态智能体框架。MetaForge将智能体行为分解为四个耦合阶段:判断(判定是否需要工具调用)、检索(选择适用工具)、适应(将工具参数锚定至任务上下文)与锻造(在线合成新技能并将其回收至工具库以供复用),形成闭环的"判断-检索-适应-锻造-回收"循环。统一编排策略使智能体能够在直接作答、复用现有工具或锻造新工具间自主抉择。通过强化学习联合优化调用必要性、检索准确性、执行有效性及锻造技能可复用性,并引入显式调用代价惩罚以抑制冗余调用。在12项基准测试中,MetaForge在准确性、效率与泛化能力上持续超越16个基线模型,验证了从静态工具库到按需自演进的范式转变。