Deep reinforcement learning (DRL) policies are vulnerable to unauthorized replication attacks, where an adversary exploits imitation learning to reproduce target policies from observed behavior. In this paper, we propose Constrained Randomization of Policy (CRoP) as a mitigation technique against such attacks. CRoP induces the execution of sub-optimal actions at random under performance loss constraints. We present a parametric analysis of CRoP, address the optimality of CRoP, and establish theoretical bounds on the adversarial budget and the expectation of loss. Furthermore, we report the experimental evaluation of CRoP in Atari environments under adversarial imitation, which demonstrate the efficacy and feasibility of our proposed method against policy replication attacks.

翻译：深入强化学习(DRL)政策很容易遭到未经授权的复制攻击,敌人利用模仿学习从观察到的行为中复制目标政策。在本文中,我们建议限制政策随机化(CROP),作为针对此类攻击的缓解技术。CROP促使在业绩损失限制下随机采取次优行动。我们对CROP进行参数分析,处理CROP的最佳性,确定敌对预算的理论界限和损失预期。此外,我们报告在对抗性模仿下对Atari环境中的CROP的实验性评估,这显示了我们针对政策重复攻击的拟议方法的有效性和可行性。