Unmanned vehicles able to conduct advanced operations without human intervention are being developed at a fast pace for many purposes. Not surprisingly, they are also expected to significantly change how military operations can be conducted. To leverage the potential of this new technology in a physically and logically contested environment, security risks are to be assessed and managed accordingly. Research on this topic points to autonomous cyber defence as one of the capabilities that may be needed to accelerate the adoption of these vehicles for military purposes. Here, we pursue this line of investigation by exploring reinforcement learning to train an agent that can autonomously respond to cyber attacks on unmanned vehicles in the context of a military operation. We first developed a simple simulation environment to quickly prototype and test some proof-of-concept agents for an initial evaluation. This agent was then applied to a more realistic simulation environment and finally deployed on an actual unmanned ground vehicle for even more realism. A key contribution of our work is demonstrating that reinforcement learning is a viable approach to train an agent that can be used for autonomous cyber defence on a real unmanned ground vehicle, even when trained in a simple simulation environment.

翻译：能够无需人工干预执行高级任务的无人载具正以极快的速度被开发，服务于多种用途。毫不意外，它们也被预期将显著改变军事行动的实施方式。为了在物理和逻辑上都存在对抗的环境中发挥这项新技术的潜力，必须相应地评估和管理安全风险。该主题的研究指出，自主网络防御可能是加速这类载具军事化应用所需的能力之一。本文沿此研究方向，探索使用强化学习来训练一个智能体，使其能够在军事行动背景下，对无人载具遭受的网络攻击做出自主响应。我们首先开发了一个简单的仿真环境，用于快速原型设计和测试一些概念验证智能体，以进行初步评估。随后，该智能体被应用于一个更逼真的仿真环境，并最终部署在一台真实的无人地面车辆上，以获取更高的真实性。我们工作的一个关键贡献在于证明了强化学习是一种可行的方法，可用于训练能够在真实无人地面车辆上执行自主网络防御的智能体，即使该智能体仅在一个简单的仿真环境中进行训练。