The ever-improving quality of LLMs has fueled the growth of a diverse range of downstream tasks, leading to an increased demand for AI automation and a burgeoning interest in developing foundation model (FM)-based autonomous agents. As AI agent systems tackle more complex tasks and evolve, they involve a wider range of stakeholders, including agent users, agentic system developers and deployers, and AI model developers. These systems also integrate multiple components such as AI agent workflows, RAG pipelines, prompt management, agent capabilities, and observability features. In this case, obtaining reliable outputs and answers from these agents remains challenging, necessitating a dependable execution process and end-to-end observability solutions. To build reliable AI agents and LLM applications, it is essential to shift towards designing AgentOps platforms that ensure observability and traceability across the entire development-to-production life-cycle. To this end, we conducted a rapid review and identified relevant AgentOps tools from the agentic ecosystem. Based on this review, we provide an overview of the essential features of AgentOps and propose a comprehensive overview of observability data/traceable artifacts across the agent production life-cycle. Our findings provide a systematic overview of the current AgentOps landscape, emphasizing the critical role of observability/traceability in enhancing the reliability of autonomous agent systems.

翻译：大型语言模型（LLM）性能的持续提升推动了多样化下游任务的增长，进而催生了对人工智能自动化的更高需求，并激发了基于基础模型（FM）的自主智能体开发热潮。随着人工智能智能体系统处理日益复杂的任务并持续演进，其涉及的利益相关方范围不断扩大，包括智能体用户、智能体系统开发与部署者以及人工智能模型开发者。这些系统还整合了多个组件，例如智能体工作流、检索增强生成（RAG）管道、提示词管理、智能体能力模块及可观测性功能。在此背景下，从这些智能体获取可靠输出与答案仍具挑战性，亟需建立可信赖的执行流程与端到端的可观测性解决方案。为构建可靠的人工智能智能体与LLM应用，必须转向设计能够保障开发至生产全生命周期可观测性与可追溯性的AgentOps平台。为此，我们通过快速文献综述从智能体生态中识别出相关AgentOps工具。基于此综述，我们概述了AgentOps的核心功能特征，并提出了覆盖智能体生产全生命周期的可观测性数据/可追溯制品的系统性框架。本研究结果系统梳理了当前AgentOps的发展格局，强调了可观测性/可追溯性在提升自主智能体系统可靠性中的关键作用。