Large language model (LLM)-based computer-use agents represent a convergence of AI and OS capabilities, enabling natural language to control system- and application-level functions. However, due to LLMs' inherent uncertainty issues, granting agents control over computers poses significant security risks. When agent actions deviate from user intentions, they can cause irreversible consequences. Existing mitigation approaches, such as user confirmation and LLM-based dynamic action validation, still suffer from limitations in usability, security, and performance. To address these challenges, we propose CSAgent, a system-level, static policy-based access control framework for computer-use agents. To bridge the gap between static policy and dynamic context and user intent, CSAgent introduces intent- and context-aware policies, and provides an automated toolchain to assist developers in constructing and refining them. CSAgent enforces these policies through an optimized OS service, ensuring that agent actions can only be executed under specific user intents and contexts. CSAgent supports protecting agents that control computers through diverse interfaces, including API, CLI, and GUI. We implement and evaluate CSAgent, which successfully defends against all attacks in the benchmarks while introducing only 1.99% performance overhead and 5.42% utility decrease.

翻译：基于大语言模型（LLM）的计算机使用代理代表了人工智能与操作系统能力的融合，使得自然语言能够控制系统级和应用级功能。然而，由于大语言模型固有的不确定性，授予代理对计算机的控制权会带来显著的安全风险。当代理行为偏离用户意图时，可能导致不可逆的后果。现有的缓解方法，例如用户确认和基于大语言模型的动态操作验证，在可用性、安全性和性能方面仍存在局限。为应对这些挑战，我们提出了CSAgent，一个面向计算机使用代理的系统级、基于静态策略的访问控制框架。为弥合静态策略与动态上下文及用户意图之间的鸿沟，CSAgent引入了意图与上下文感知策略，并提供自动化工具链以协助开发者构建和优化这些策略。CSAgent通过一个优化的操作系统服务来强制执行这些策略，确保代理操作只能在特定的用户意图和上下文中执行。CSAgent支持保护通过多种接口（包括API、CLI和GUI）控制计算机的代理。我们实现并评估了CSAgent，该系统成功防御了基准测试中的所有攻击，同时仅引入1.99%的性能开销和5.42%的效用降低。