Autonomous driving has received a great deal of attention in the automotive industry and is often seen as the future of transportation. The development of autonomous driving technology has been greatly accelerated by the growth of end-to-end machine learning techniques that have been successfully used for perception, planning, and control tasks. An important aspect of autonomous driving planning is knowing how the environment evolves in the immediate future and taking appropriate actions. An autonomous driving system should effectively use the information collected from the various sensors to form an abstract representation of the world to maintain situational awareness. For this purpose, deep learning models can be used to learn compact latent representations from a stream of incoming data. However, most deep learning models are trained end-to-end and do not incorporate any prior knowledge (e.g., from physics) of the vehicle in the architecture. In this direction, many works have explored physics-infused neural network (PINN) architectures to infuse physics models during training. Inspired by this observation, we present a Kalman filter augmented recurrent neural network architecture to learn the latent representation of the traffic flow using front camera images only. We demonstrate the efficacy of the proposed model in both imitation and reinforcement learning settings using both simulated and real-world datasets. The results show that incorporating an explicit model of the vehicle (states estimated using Kalman filtering) in the end-to-end learning significantly increases performance.

翻译：自动驾驶在汽车工业中受到了广泛关注，常被视为交通领域的未来。端到端机器学习技术的进步极大地加速了自动驾驶技术的发展，这些技术已成功应用于感知、规划和控制任务。自动驾驶规划的一个重要方面是了解环境在近期内如何演变，并采取相应行动。自动驾驶系统应有效利用从各种传感器收集的信息，形成环境的抽象表征，以维持态势感知。为此，深度学习模型可用于从输入数据流中学习紧凑的潜在表征。然而，大多数深度学习模型采用端到端训练，并未在架构中融入车辆的先验知识（如物理知识）。在此方向上，许多研究探索了物理融合神经网络（PINN）架构，以在训练过程中注入物理模型。受此启发，我们提出了一种基于卡尔曼滤波增强的循环神经网络架构，仅利用前置摄像头图像学习交通流的潜在表征。我们在模拟和真实数据集上，通过模仿学习和强化学习两种设置验证了所提模型的有效性。结果表明，在端到端学习过程中融入显式车辆模型（通过卡尔曼滤波估计的状态）可显著提升性能。