Memory Gym presents a suite of 2D partially observable environments, namely Mortar Mayhem, Mystery Path, and Searing Spotlights, designed to benchmark memory capabilities in decision-making agents. These environments, originally with finite tasks, are expanded into innovative, endless formats, mirroring the escalating challenges of cumulative memory games such as ``I packed my bag''. This progression in task design shifts the focus from merely assessing sample efficiency to also probing the levels of memory effectiveness in dynamic, prolonged scenarios. To address the gap in available memory-based Deep Reinforcement Learning baselines, we introduce an implementation that integrates Transformer-XL (TrXL) with Proximal Policy Optimization. This approach utilizes TrXL as a form of episodic memory, employing a sliding window technique. Our comparative study between the Gated Recurrent Unit (GRU) and TrXL reveals varied performances across different settings. TrXL, on the finite environments, demonstrates superior sample efficiency in Mystery Path and outperforms in Mortar Mayhem. However, GRU is more efficient on Searing Spotlights. Most notably, in all endless tasks, GRU makes a remarkable resurgence, consistently outperforming TrXL by significant margins. Website and Source Code: https://github.com/MarcoMeter/endless-memory-gym/

翻译：《记忆健身房》提出了一套二维部分可观测环境，即迫击炮混乱、神秘路径和灼热聚光灯，旨在基准测试决策智能体的记忆能力。这些原本具有有限任务的环境被扩展为创新的无尽格式，类似于累计记忆游戏（如“我打包了我的包”）中不断升级的挑战。这种任务设计的演进将焦点从单纯评估样本效率转向同时探测动态、长期场景中记忆有效性的层次。为填补现有基于记忆的深度强化学习基准的空白，我们引入了一种集成Transformer-XL（TrXL）与近端策略优化的实现。该方法利用TrXL作为情景记忆的一种形式，采用滑动窗口技术。我们通过门控循环单元（GRU）与TrXL的比较研究揭示了在不同设置下的性能差异。在有限环境中，TrXL在神秘路径上展现出卓越的样本效率，并在迫击炮混乱中表现更优。然而，GRU在灼热聚光灯上更为高效。最值得注意的是，在所有无尽任务中，GRU显著复苏，持续以大幅优势超越TrXL。网站与源代码：https://github.com/MarcoMeter/endless-memory-gym/