Style Transfer has been proposed in a number of fields: fine arts, natural language processing, and fixed trajectories. We scale this concept up to control policies within a Deep Reinforcement Learning infrastructure. Each network is trained to maximize the expected reward, which typically encodes the goal of an action, and can be described as the content. The expressive power of deep neural networks enables encoding a secondary task, which can be described as the style. The Neural Policy Style Transfer (NPST) algorithm is proposed to transfer the style of one policy to another, while maintaining the content of the latter. Different policies are defined via Deep Q-Network architectures. These models are trained using demonstrations through Inverse Reinforcement Learning. Two different sets of user demonstrations are performed, one for content and other for style. Different styles are encoded as defined by user demonstrations. The generated policy is the result of feeding a content policy and a style policy to the NPST algorithm. Experiments are performed in a catch-ball game inspired by the Deep Reinforcement Learning classical Atari games; and a real-world painting scenario with a full-sized humanoid robot, based on previous works of the authors. The implementation of three different Q-Network architectures (Shallow, Deep and Deep Recurrent Q-Network) to encode the policies within the NPST framework is proposed and the results obtained in the experiments with each of these architectures compared.

翻译：风格迁移已在多个领域被提出：美术、自然语言处理以及固定轨迹。我们将这一概念扩展到深度强化学习框架内的控制策略。每个网络被训练以最大化预期奖励，该奖励通常编码了动作的目标，可被描述为内容。深度神经网络的表达能力使其能够编码可被描述为风格的辅助任务。本文提出神经策略风格迁移（NPST）算法，用于将一种策略的风格迁移到另一种策略上，同时保持后者的内容。不同策略通过深度Q网络架构定义。这些模型通过逆强化学习使用示范数据进行训练。用户执行两组不同的示范：一组用于内容，另一组用于风格。不同的风格由用户示范进行编码。生成的策略是将内容策略和风格策略输入NPST算法的结果。实验在受深度强化学习经典Atari游戏启发的接球游戏以及基于作者先前工作的全尺寸仿人机器人真实绘画场景中进行。我们提出在NPST框架内使用三种不同的Q网络架构（浅层、深层和深度循环Q网络）来编码策略，并比较了每种架构在实验中取得的结果。