LLM-based intelligent agents face significant deployment challenges, particularly related to resource management. Allowing unrestricted access to LLM or tool resources can lead to inefficient or even potentially harmful resource allocation and utilization for agents. Furthermore, the absence of proper scheduling and resource management mechanisms in current agent designs hinders concurrent processing and limits overall system efficiency. As the diversity and complexity of agents continue to grow, addressing these resource management issues becomes increasingly critical to LLM-based agent systems. To address these challenges, this paper proposes the architecture of AIOS (LLM-based AI Agent Operating System) under the context of managing LLM-based agents. It introduces a novel architecture for serving LLM-based agents by isolating resources and LLM-specific services from agent applications into an AIOS kernel. This AIOS kernel provides fundamental services (e.g., scheduling, context management, memory management, storage management, access control) and efficient management of resources (e.g., LLM and external tools) for runtime agents. To enhance usability, AIOS also includes an AIOS-Agent SDK, a comprehensive suite of APIs designed for utilizing functionalities provided by the AIOS kernel. Experimental results demonstrate that using AIOS can achieve up to 2.1x faster execution for serving agents built by various agent frameworks. The source code is available at https://github.com/agiresearch/AIOS.

翻译：基于大语言模型（LLM）的智能体在部署过程中面临显著的资源管理挑战。若允许智能体无限制地访问LLM或工具资源，可能导致低效甚至潜在有害的资源分配与利用。此外，当前智能体设计缺乏适当的调度与资源管理机制，阻碍了并发处理能力并限制了整体系统效率。随着智能体多样性与复杂性的持续增长，解决这些资源管理问题对基于LLM的智能体系统愈发关键。为应对这些挑战，本文在管理基于LLM的智能体背景下，提出了AIOS（基于LLM的AI智能体操作系统）架构。该架构通过将资源及LLM专属服务从智能体应用中隔离至AIOS内核，为基于LLM的智能体服务提供了一种新颖的架构方案。AIOS内核为运行时智能体提供基础服务（如调度、上下文管理、内存管理、存储管理、访问控制）以及对资源（如LLM与外部工具）的高效管理。为提升易用性，AIOS还包含AIOS-Agent SDK——一套为利用AIOS内核功能而设计的完整API工具集。实验结果表明，使用AIOS可为各类智能体框架构建的智能体服务实现最高2.1倍的执行加速。源代码已发布于https://github.com/agiresearch/AIOS。