Vision-based autonomous urban driving in dense traffic is quite challenging due to the complicated urban environment and the dynamics of the driving behaviors. Widely-applied methods either heavily rely on hand-crafted rules or learn from limited human experience, which makes them hard to generalize to rare but critical scenarios. In this paper, we present a novel CAscade Deep REinforcement learning framework, CADRE, to achieve model-free vision-based autonomous urban driving. In CADRE, to derive representative latent features from raw observations, we first offline train a Co-attention Perception Module (CoPM) that leverages the co-attention mechanism to learn the inter-relationships between the visual and control information from a pre-collected driving dataset. Cascaded by the frozen CoPM, we then present an efficient distributed proximal policy optimization framework to online learn the driving policy under the guidance of particularly designed reward functions. We perform a comprehensive empirical study with the CARLA NoCrash benchmark as well as specific obstacle avoidance scenarios in autonomous urban driving tasks. The experimental results well justify the effectiveness of CADRE and its superiority over the state-of-the-art by a wide margin.

翻译：在密集交通中基于视觉的自主城市驾驶由于复杂的城市环境和驾驶行为的动态性而极具挑战性。广泛应用的方法要么严重依赖手工设计的规则，要么从有限的人类经验中学习，这使得它们难以泛化到罕见但关键的场景。本文提出了一种新型级联深度强化学习框架CADRE，以实现无模型的基于视觉的自主城市驾驶。在CADRE中，为从原始观测中提取代表性潜在特征，我们首先离线训练一个协同注意力感知模块（CoPM），该模块利用协同注意力机制从预先收集的驾驶数据集中学习视觉信息与控制信息之间的相互关系。级联冻结的CoPM后，我们进一步提出一种高效的分布式近端策略优化框架，在特别设计的奖励函数引导下在线学习驾驶策略。我们利用CARLA NoCrash基准测试以及自主城市驾驶任务中的特定避障场景进行了全面的实证研究。实验结果充分验证了CADRE的有效性及其相对于现有最先进方法的显著优越性。