To achieve fully autonomous driving, vehicles must be capable of continuously performing various driving tasks, including lane keeping and car following, both of which are fundamental and well-studied driving ones. However, previous studies have mainly focused on individual tasks, and car following tasks have typically relied on complete leader-follower information to attain optimal performance. To address this limitation, we propose a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers. To evaluate the performance of our DRL agent, we compare it with a baseline controller and use various performance metrics for quantitative analysis. Furthermore, we conduct a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent. To the best of our knowledge, our vision-based car following and lane keeping agent with Sim2Real transfer capability is the first of its kind.

翻译：为实现完全自动驾驶，车辆需持续执行多种驾驶任务，包括车道保持与跟车行驶——这两项均为已得到充分研究的基础性驾驶操作。然而，现有研究主要聚焦于单一任务，其中跟车任务通常依赖完整的领导者-跟随者信息才能达到最优性能。为突破这一局限，我们提出一种基于视觉的深度强化学习智能体，该智能体可同时执行车道保持与跟车操作。为评估所提出DRL智能体的性能，我们将其与基准控制器进行对比，并采用多种性能指标进行量化分析。此外，我们通过真实场景评估验证了训练后DRL智能体的Sim2Real迁移能力。据我们所知，本研究提出的具备Sim2Real迁移能力的视觉驱动车道保持与跟车智能体属首创性成果。