Legged locomotion has recently achieved remarkable success with the progress of machine learning techniques, especially deep reinforcement learning (RL). Controllers employing neural networks have demonstrated empirical and qualitative robustness against real-world uncertainties, including sensor noise and external perturbations. However, formally investigating the vulnerabilities of these locomotion controllers remains a challenge. This difficulty arises from the requirement to pinpoint vulnerabilities across a long-tailed distribution within a high-dimensional, temporally sequential space. As a first step towards quantitative verification, we propose a computational method that leverages sequential adversarial attacks to identify weaknesses in learned locomotion controllers. Our research demonstrates that, even state-of-the-art robust controllers can fail significantly under well-designed, low-magnitude adversarial sequence. Through experiments in simulation and on the real robot, we validate our approach's effectiveness, and we illustrate how the results it generates can be used to robustify the original policy and offer valuable insights into the safety of these black-box policies.

翻译：腿式运动近年来随着机器学习技术，尤其是深度强化学习的进步取得了显著成功。采用神经网络的控制已展现出对现实世界不确定性（包括传感器噪声和外部扰动）的经验性和定性鲁棒性。然而，正式探究这些运动控制器的脆弱性仍具挑战。这一难点源于需要在高维时间序列空间中，从长尾分布中精确定位脆弱点。作为迈向定量验证的第一步，我们提出一种计算方法，利用序列对抗攻击识别学习型运动控制器的弱点。研究表明，即使是当前最先进的鲁棒控制器，在精心设计的低幅度对抗序列下也可能显著失效。通过仿真和真实机器人实验，我们验证了该方法的有效性，并展示了生成结果如何用于强化原始策略，为这些黑盒策略的安全性提供宝贵见解。