Technological trends show that Radio Frequency Reinforcement Learning (RFRL) will play a prominent role in the wireless communication systems of the future. Applications of RFRL range from military communications jamming to enhancing WiFi networks. Before deploying algorithms for these purposes, they must be trained in a simulation environment to ensure adequate performance. For this reason, we previously created the RFRL Gym: a standardized, accessible tool for the development and testing of reinforcement learning (RL) algorithms in the wireless communications space. This environment leveraged the OpenAI Gym framework and featured customizable simulation scenarios within the RF spectrum. However, the RFRL Gym was limited to training a single RL agent per simulation; this is not ideal, as most real-world RF scenarios will contain multiple intelligent agents in cooperative, competitive, or mixed settings, which is a natural consequence of spectrum congestion. Therefore, through integration with Ray RLlib, multi-agent reinforcement learning (MARL) functionality for training and assessment has been added to the RFRL Gym, making it even more of a robust tool for RF spectrum simulation. This paper provides an overview of the updated RFRL Gym environment. In this work, the general framework of the tool is described relative to comparable existing resources, highlighting the significant additions and refactoring we have applied to the Gym. Afterward, results from testing various RF scenarios in the MARL environment and future additions are discussed.

翻译：技术发展趋势表明，射频强化学习（RFRL）将在未来无线通信系统中发挥重要作用。RFRL的应用范围涵盖从军事通信干扰到增强WiFi网络等多个领域。在部署用于这些目的的算法之前，必须在仿真环境中对其进行训练以确保足够的性能。为此，我们先前创建了RFRL Gym：一个用于在无线通信领域开发和测试强化学习（RL）算法的标准化、易用工具。该环境基于OpenAI Gym框架构建，并支持在射频频谱内进行可定制的仿真场景。然而，RFRL Gym仅限于每个仿真训练单个RL智能体；这并非理想情况，因为大多数现实世界的射频场景都包含多个智能体处于协作、竞争或混合设置中，这是频谱拥塞的自然结果。因此，通过与Ray RLlib集成，我们为RFRL Gym增加了用于训练和评估的多智能体强化学习（MARL）功能，使其成为更加强大的射频频谱仿真工具。本文概述了更新后的RFRL Gym环境。在这项工作中，我们描述了该工具相对于现有可比资源的一般框架，重点介绍了我们对Gym进行的重要功能添加和重构。随后，讨论了在MARL环境中测试各种射频场景的结果以及未来的功能扩展方向。