Autonomous systems are emerging in many application domains. With the recent advancements in artificial intelligence and machine learning, sensor technology, perception algorithms and robotics, scenarios previously requiring strong human involvement can be handled by autonomous systems. With the independence from human control, cybersecurity of such systems becomes even more critical as no human intervention in case of undesired behavior is possible. In this context, this paper discusses emerging security challenges in autonomous systems design which arise in many domains such as autonomous incident response, risk assessment, data availability, systems interaction, trustworthiness, updatability, access control, as well as the reliability and explainability of machine learning methods. In all these areas, this paper thoroughly discusses the state of the art, identifies emerging security challenges and proposes research directions to address these challenges for developing secure autonomous systems.

翻译：自主系统正在诸多应用领域中兴起。随着人工智能与机器学习、传感器技术、感知算法及机器人技术的最新进展，以往需要大量人类参与的场景如今可由自主系统处理。由于脱离人类控制，此类系统的网络安全变得更为关键，因为一旦出现非预期行为，人类无法进行干预。在此背景下，本文探讨了自主系统设计中新兴的安全挑战，这些挑战涉及多个领域，包括自主应急响应、风险评估、数据可用性、系统交互、可信性、可更新性、访问控制，以及机器学习方法的可靠性与可解释性。针对所有这些方面，本文深入讨论了现有技术水平，识别了新兴的安全挑战，并提出了应对这些挑战的研究方向，以开发安全的自主系统。