Deep reinforcement learning has the potential to address various scientific problems. In this paper, we implement an optics simulation environment for reinforcement learning based controllers. The environment captures the essence of nonconvexity, nonlinearity, and time-dependent noise inherent in optical systems, offering a more realistic setting. Subsequently, we provide the benchmark results of several reinforcement learning algorithms on the proposed simulation environment. The experimental findings demonstrate the superiority of off-policy reinforcement learning approaches over traditional control algorithms in navigating the intricacies of complex optical control environments. The code of the paper is available at https://github.com/Walleclipse/Reinforcement-Learning-Pulse-Stacking.

翻译：深度强化学习有潜力解决各类科学问题。本文针对基于强化学习的控制器实现了一种光学仿真环境。该环境捕捉了光学系统中固有的非凸性、非线性和时变噪声的本质，提供了更真实的实验设置。随后，我们在所提出的仿真环境下给出了多种强化学习算法的基准测试结果。实验结果表明，在应对复杂光学控制环境的复杂性时，离策略强化学习方法相较于传统控制算法具有优越性。本文代码发布在 https://github.com/Walleclipse/Reinforcement-Learning-Pulse-Stacking。