Reinforcement Learning (RL) algorithms exhibit high sample complexity, particularly when applied to Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). As a response, projects such as SampleFactory, EnvPool, Brax, and IsaacLab migrate parallel execution of classic environments such as MuJoCo and Atari into C++ thread pools or the GPU to decrease the computational cost of environment steps. We are interested in optimizing the decision-level of human-AI joint operations, so we introduce a compute-efficient Dec-POMDP engine natively architected in C++ called Hide-And-Seek-Engine. By employing Data-Oriented Design (DOD) principles, explicit 64-byte cache-line alignment to remove false sharing, and a zero-copy PyTorch memory bridge using pinned memory and Direct Memory Access (DMA), our engine sustains throughput of up to 33,000,000 steps per second (SPS) in a single-agent, 1024-environment, decentralized observations on an AMD Ryzen 9950X (16 cores). Ten agents reduces FPS to 7M SPS with generating random actions contributing 1/3rd the total runtime for reference. The engine achieves a throughput increase of approximately 3,500$\times$ over the baseline single threaded vectorized NumPy implementation and successfully trains cooperative multi-agent policies via PPO, DQN, and SAC in minutes, validating both its performance and generality.

翻译：强化学习（RL）算法具有较高的样本复杂度，特别是在应用于去中心化部分可观测马尔可夫决策过程（Dec-POMDP）时。为此，诸如SampleFactory、EnvPool、Brax和IsaacLab等项目将MuJoCo和Atari等经典环境的并行执行迁移至C++线程池或GPU，以降低环境步骤的计算成本。我们专注于优化人机联合操作中的决策层级，因此提出了一种原生采用C++架构的计算高效Dec-POMDP引擎，称为捉迷藏引擎（Hide-And-Seek-Engine）。通过采用数据导向设计（DOD）原则、显式64字节缓存行对齐以消除伪共享，以及基于固定内存和直接内存访问（DMA）的零拷贝PyTorch内存桥，我们的引擎在单智能体、1024环境、去中心化观测条件下，在AMD Ryzen 9950X（16核）上维持高达每秒33,000,000步（SPS）的吞吐量。十个智能体时，吞吐量降至7M SPS，其中生成随机动作约占运行总时长的三分之一。该引擎相较于基线单线程向量化NumPy实现，吞吐量提升了约3,500倍，并能在数分钟内通过PPO、DQN和SAC成功训练协作多智能体策略，验证了其性能与通用性。