The transition of Multi-Agent Reinforcement Learning (MARL) policies from simulated cyber wargames to operational Security Operations Centers (SOCs) is fundamentally bottlenecked by the Sim2Real gap. Legacy simulators abstract away network protocol physics, rely on synchronous ticks, and provide clean state vectors rather than authentic, noisy telemetry. To resolve these limitations, we introduce NetForge_RL: a high-fidelity cyber operations simulator that reformulates network defense as an asynchronous, continuous-time Partially Observable Semi-Markov Decision Process (POSMDP). NetForge enforces Zero-Trust Network Access (ZTNA) constraints and requires defenders to process NLP-encoded SIEM telemetry. Crucially, NetForge bridges the Sim2Real gap natively via a dual-mode engine, allowing high-throughput MARL training in a mock hypervisor and zero-shot evaluation against live exploits in a Docker hypervisor. To navigate this continuous-time POSMDP, we propose Continuous-Time Graph MARL (CT-GMARL), utilizing fixed-step Neural Ordinary Differential Equations (ODEs) to process irregularly sampled alerts. We evaluate our framework against discrete baselines (R-MAPPO, QMIX). Empirical results demonstrate that CT-GMARL achieves a converged median Blue reward of 57,135 - a 2.0x improvement over R-MAPPO and 2.1x over QMIX. Critically, CT-GMARL restores 12x more compromised services than the strongest baseline by avoiding the "scorched earth" failure mode of trivially minimizing risk by destroying network utility. On zero-shot transfer to the live Docker environment, CT-GMARL policies achieve a median reward of 98,026, validating the Sim2Real bridge.

翻译：将多智能体强化学习（MARL）策略从模拟网络兵棋推演迁移至实际安全运营中心（SOC）的根本瓶颈在于Sim2Real差距。传统模拟器抽象了网络协议物理特性，依赖同步时钟节拍，并提供简洁状态向量而非真实且有噪声的遥测数据。为解决这些局限，我们提出NetForge_RL：一个高保真网络运营模拟器，将网络防御重新表述为异步、连续时间的部分可观测半马尔可夫决策过程（POSMDP）。NetForge强制执行零信任网络访问（ZTNA）约束，并要求防御者处理经自然语言处理（NLP）编码的SIEM遥测数据。关键的是，NetForge通过双模引擎原生弥合Sim2Real差距，允许在模拟虚拟机监控器中进行高通量MARL训练，并在Docker虚拟机监控器中针对实时漏洞利用进行零样本评估。为导航此连续时间POSMDP，我们提出连续时间图MARL（CT-GMARL），利用固定步长神经常微分方程（ODE）处理不规则采样的告警。我们针对离散基线方法（R-MAPPO、QMIX）评估了框架。实验结果表明，CT-GMARL的中位收敛蓝方奖励为57,135——比R-MAPPO提升2.0倍，比QMIX提升2.1倍。关键的是，CT-GMARL通过避免因简单破坏网络效用而最小化风险的“焦土”失效模式，恢复的被攻陷服务数量比最强基线多12倍。在向实时Docker环境的零样本迁移中，CT-GMARL策略的中位奖励达98,026，验证了Sim2Real桥梁的有效性。