Reinforcement Learning (RL) is a promising solution, allowing Unmanned Underwater Vehicles (UUVs) to learn optimal behaviors through trial and error. However, existing simulators lack efficient integration with RL methods, limiting training scalability and performance. This paper introduces MarineGym, a novel simulation framework designed to enhance RL training efficiency for UUVs by utilizing GPU acceleration. MarineGym offers a 10,000-fold performance improvement over real-time simulation on a single GPU, enabling rapid training of RL algorithms across multiple underwater tasks. Key features include realistic dynamic modeling of UUVs, parallel environment execution, and compatibility with popular RL frameworks like PyTorch and TorchRL. The framework is validated through four distinct tasks: station-keeping, circle tracking, helical tracking, and lemniscate tracking. This framework sets the stage for advancing RL in underwater robotics and facilitating efficient training in complex, dynamic environments.

翻译：强化学习（Reinforcement Learning, RL）是一种前景广阔的解决方案，它使得无人水下航行器（Unmanned Underwater Vehicles, UUVs）能够通过试错学习最优行为。然而，现有仿真器缺乏与RL方法的高效集成，限制了训练的可扩展性和性能。本文介绍了MarineGym，一种新颖的仿真框架，旨在通过利用GPU加速来提升UUV的RL训练效率。MarineGym在单GPU上实现了相对于实时仿真高达10,000倍的性能提升，从而能够在多种水下任务中快速训练RL算法。其关键特性包括：UUV的高保真动力学建模、并行环境执行，以及与PyTorch、TorchRL等主流RL框架的兼容性。该框架通过四项不同任务进行了验证：定点保持、圆周跟踪、螺旋线跟踪和双纽线跟踪。本框架为推进水下机器人领域的RL研究，以及在复杂动态环境中实现高效训练奠定了基础。