Advancements in Large Language Models (LLMs) are revolutionizing the development of autonomous agentic systems by enabling dynamic, context-aware task decomposition and automated tool selection. These sophisticated systems possess significant automation potential across various industries, managing complex tasks, interacting with external systems to enhance knowledge, and executing actions independently. This paper presents three primary contributions to advance this field: - Advanced Agentic Framework: A system that handles multi-hop queries, generates and executes task graphs, selects appropriate tools, and adapts to real-time changes. - Novel Evaluation Metrics: Introduction of Node F1 Score, Structural Similarity Index (SSI), and Tool F1 Score to comprehensively assess agentic systems. - Specialized Dataset: Development of an AsyncHow-based dataset for analyzing agent behavior across different task complexities. Our findings reveal that asynchronous and dynamic task graph decomposition significantly enhances system responsiveness and scalability, particularly for complex, multi-step tasks. Detailed analysis shows that structural and node-level metrics are crucial for sequential tasks, while tool-related metrics are more important for parallel tasks. Specifically, the Structural Similarity Index (SSI) is the most significant predictor of performance in sequential tasks, and the Tool F1 Score is essential for parallel tasks. These insights highlight the need for balanced evaluation methods that capture both structural and operational dimensions of agentic systems. Additionally, our evaluation framework, validated through empirical analysis and statistical testing, provides valuable insights for improving the adaptability and reliability of agentic systems in dynamic environments.
翻译:大型语言模型(LLM)的进展正在通过实现动态、情境感知的任务分解与自动化工具选择,彻底改变自主智能体系统的开发。这些复杂系统在各行业具有巨大的自动化潜力,能够管理复杂任务、与外部系统交互以扩展知识,并独立执行操作。本文提出推动该领域发展的三项核心贡献:- 先进智能体框架:一种能够处理多跳查询、生成并执行任务图、选择合适工具并适应实时变化的系统。- 新型评估指标:引入节点F1分数、结构相似性指数(SSI)和工具F1分数,以全面评估智能体系统。- 专用数据集:开发基于AsyncHow的数据集,用于分析不同任务复杂度下的智能体行为。研究发现,异步动态任务图分解显著提升了系统响应能力与可扩展性,尤其适用于复杂的多步骤任务。详细分析表明,结构与节点级指标对顺序任务至关重要,而工具相关指标对并行任务更为重要。具体而言,结构相似性指数(SSI)是顺序任务性能的最关键预测因子,工具F1分数则是并行任务的核心评估要素。这些发现凸显了需要采用能同时捕捉智能体系统结构维度与操作维度的平衡评估方法。此外,通过实证分析与统计检验验证的评估框架,为提升智能体系统在动态环境中的适应性与可靠性提供了重要参考。