Reinforcement Learning can be applied to various tasks, and environments. Many of these environments have a similar shared structure, which can be exploited to improve RL performance on other tasks. Transfer learning can be used to take advantage of this shared structure, by learning policies that are transferable across different tasks and environments and can lead to more efficient learning as well as improved performance on a wide range of tasks. This work explores as well as compares the performance between RL models being trained from the scratch and on different approaches of transfer learning. Additionally, the study explores the performance of a model trained on multiple game environments, with the goal of developing a universal game-playing agent as well as transfer learning a pre-trained encoder using DQN, and training it on the same game or a different game. Our DQN model achieves a mean episode reward of 46.16 which even beats the human-level performance with merely 20k episodes which is significantly lower than deepmind's 1M episodes. The achieved mean rewards of 533.42 and 402.17 on the Assault and Space Invader environments respectively, represent noteworthy performance on these challenging environments.
翻译:强化学习可应用于多种任务与环境,其中许多环境具有相似的共享结构,这种结构可被利用来提升其他任务上的强化学习性能。迁移学习能够利用这种共享结构,通过学习可跨不同任务和环境迁移的策略,从而在广泛任务上实现更高效的学习与性能提升。本研究探索并比较了从零训练的强化学习模型与采用不同迁移学习方法的表现。此外,研究还考察了在多游戏环境中训练的模型性能,旨在开发通用游戏智能体,同时探索使用DQN预训练编码器进行迁移学习,并在相同或不同游戏上对其进行训练。我们的DQN模型在仅2万幕(远低于DeepMind的100万幕)的训练中即达到平均单幕奖励46.16,甚至超越人类水平表现。在Assault和Space Invader环境中分别取得的533.42和402.17平均奖励值,代表了这些挑战性环境上的卓越性能。