The dynamic nature of driving environments and the presence of diverse road users pose significant challenges for decision-making in autonomous driving. Deep reinforcement learning (DRL) has emerged as a popular approach to tackle this problem. However, the application of existing DRL solutions is mainly confined to simulated environments due to safety concerns, impeding their deployment in real-world. To overcome this limitation, this paper introduces a novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logics (DRLSL) that combines the strengths of DRL (learning from experience) and symbolic first-order logics knowledge-driven reasoning) to enable safe learning in real-time interactions of autonomous driving within real environments. This innovative approach provides a means to learn autonomous driving policies by actively engaging with the physical environment while ensuring safety. We have implemented the DRLSL framework in autonomous driving using the highD dataset and demonstrated that our method successfully avoids unsafe actions during both the training and testing phases. Furthermore, our results indicate that DRLSL achieves faster convergence during training and exhibits better generalizability to new driving scenarios compared to traditional DRL methods.

翻译：驾驶环境的动态特性以及多元道路使用者的存在，给自动驾驶决策带来了重大挑战。深度强化学习已成为解决该问题的常用方法。然而，现有的深度强化学习方案由于安全问题主要局限于模拟环境，阻碍了其在现实世界中的部署。为克服这一局限，本文提出了一种新颖的神经符号无模型深度强化学习方法，即带有符号逻辑的深度强化学习（DRSL），该方法结合了深度强化学习（从经验中学习）与符号一阶逻辑（知识驱动推理）的优势，能够在现实环境中实现自动驾驶实时交互中的安全学习。这一创新方法通过主动与物理环境交互来学习自动驾驶策略，同时确保安全性。我们利用highD数据集在自动驾驶中实现了DRSL框架，并证明该方法在训练和测试阶段均能成功规避不安全行为。此外，我们的结果表明，与传统深度强化学习方法相比，DRSL在训练过程中收敛速度更快，且对新驾驶场景表现出更强的泛化能力。