Trajectory planning is vital for autonomous driving, ensuring safe and efficient navigation in complex environments. While recent learning-based methods, particularly reinforcement learning (RL), have shown promise in specific scenarios, RL planners struggle with training inefficiencies and managing large-scale, real-world driving scenarios. In this paper, we introduce \textbf{CarPlanner}, a \textbf{C}onsistent \textbf{a}uto-\textbf{r}egressive \textbf{Planner} that uses RL to generate multi-modal trajectories. The auto-regressive structure enables efficient large-scale RL training, while the incorporation of consistency ensures stable policy learning by maintaining coherent temporal consistency across time steps. Moreover, CarPlanner employs a generation-selection framework with an expert-guided reward function and an invariant-view module, simplifying RL training and enhancing policy performance. Extensive analysis demonstrates that our proposed RL framework effectively addresses the challenges of training efficiency and performance enhancement, positioning CarPlanner as a promising solution for trajectory planning in autonomous driving. To the best of our knowledge, we are the first to demonstrate that the RL-based planner can surpass both IL- and rule-based state-of-the-arts (SOTAs) on the challenging large-scale real-world dataset nuPlan. Our proposed CarPlanner surpasses RL-, IL-, and rule-based SOTA approaches within this demanding dataset.

翻译：轨迹规划对于自动驾驶至关重要，它确保了在复杂环境中安全高效的导航。尽管近年来基于学习的方法，特别是强化学习（RL），在特定场景中展现出潜力，但RL规划器仍面临训练效率低下以及难以处理大规模真实世界驾驶场景的挑战。本文提出\textbf{CarPlanner}，一种\textbf{一致性自回归规划器}，它利用RL生成多模态轨迹。其自回归结构实现了高效的大规模RL训练，而一致性机制的引入则通过保持跨时间步的连贯时间一致性，确保了策略学习的稳定性。此外，CarPlanner采用了一个生成-选择框架，该框架包含专家引导的奖励函数和一个不变视角模块，从而简化了RL训练并提升了策略性能。广泛的分析表明，我们提出的RL框架有效解决了训练效率和性能提升的挑战，使CarPlanner成为自动驾驶轨迹规划领域一个有前景的解决方案。据我们所知，我们首次证明了基于RL的规划器能够在具有挑战性的大规模真实世界数据集nuPlan上超越基于模仿学习（IL）和基于规则的最先进方法（SOTA）。我们提出的CarPlanner在该高要求数据集上超越了基于RL、IL和规则的SOTA方法。