Frontier AI systems are increasingly capable of cybersecurity tasks, including codebase inspection, vulnerability detection, and exploitation. However, evaluating their offensive capabilities remains constrained by limited access to open, reproducible, multi-host cyber ranges. Existing public benchmarks capture isolated skills such as CTF solving, vulnerability reproduction, and exploit generation, but often abstract away realistic intrusion workflows: discovering exposed services, gaining a foothold, collecting internal information, and expanding compromise across hosts. This gap makes it difficult to observe emerging risks early, because frontier AI systems are rarely evaluated under realistic attack conditions. We introduce AgentCyberRange, the first open, multi-range infrastructure for measuring autonomous cyber attack capability in realistic cyber ranges. It combines 110 vulnerabilities across 15 real web applications and 8 enterprise-like cyber ranges with 156 internal hosts, plus Cage, a toolchain for execution, orchestration, result collection, and verification. The benchmark covers two core stages: web exploitation, where agents explore exposed applications and validate vulnerabilities, and post exploitation, where agents turn an initial foothold into broader internal compromise. We evaluate six frontier AI systems under matched prompts and budgets. GPT-5.5 with Codex performs best, solving 16.1% of web exploitation tasks and 31.7% of post-exploitation tasks; with more concrete hints, these rates increase to 33.0% and 46.3%. We also observe out-of-benchmark findings, including unknown vulnerabilities in popular projects, and payload mutation that bypasses host defenses. These results show that open cyber-range evaluation is necessary for observing emerging offensive capabilities under realistic and reproducible conditions.

翻译：前沿人工智能系统在网络安全任务上的能力日益增强，包括代码库审查、漏洞检测与利用。然而，由于缺乏开放、可复现的多主机网络靶场，对其攻击能力的评估仍受到限制。现有公开基准测试仅聚焦于CTF解题、漏洞复现和利用生成等孤立技能，往往忽略了真实的入侵工作流程：发现暴露的服务、获取立足点、收集内部信息以及跨主机扩展攻击范围。这一空白使得前沿AI系统很少在真实攻击条件下接受评估，从而难以早期观察新兴风险。我们提出AgentCyberRange——首个在真实网络靶场中衡量自主网络攻击能力的开放、多靶场基础设施。它整合了15个真实Web应用中的110个漏洞、8个包含156台内部主机的企业级网络靶场，以及用于执行、编排、结果收集与验证的工具链Cage。该基准测试涵盖两个核心阶段：Web利用阶段（智能体探索暴露的应用并验证漏洞）和后利用阶段（智能体将初始立足点转化为更广泛的内部入侵）。我们在匹配的提示词和预算下评估了六个前沿AI系统。GPT-5.5 with Codex表现最佳，解决了16.1%的Web利用任务和31.7%的后利用任务；在更具体的提示下，这些比率分别提升至33.0%和46.3%。我们还观察到基准测试之外的发现，包括流行项目中的未知漏洞以及绕过主机防御的载荷变异。这些结果表明，开放网络靶场评估对于在真实且可复现的条件下观察新兴攻击能力至关重要。