To address the intricate challenges of decentralized cooperative scheduling and motion planning in Autonomous Mobility-on-Demand (AMoD) systems, this paper introduces LMMCoDrive, a novel cooperative driving framework that leverages a Large Multimodal Model (LMM) to enhance traffic efficiency in dynamic urban environments. This framework seamlessly integrates scheduling and motion planning processes to ensure the effective operation of Cooperative Autonomous Vehicles (CAVs). The spatial relationship between CAVs and passenger requests is abstracted into a Bird's-Eye View (BEV) to fully exploit the potential of the LMM. Besides, trajectories are cautiously refined for each CAV while ensuring collision avoidance through safety constraints. A decentralized optimization strategy, facilitated by the Alternating Direction Method of Multipliers (ADMM) within the LMM framework, is proposed to drive the graph evolution of CAVs. Simulation results demonstrate the pivotal role and significant impact of LMM in optimizing CAV scheduling and enhancing decentralized cooperative optimization process for each vehicle. This marks a substantial stride towards achieving practical, efficient, and safe AMoD systems that are poised to revolutionize urban transportation. The code is available at https://github.com/henryhcliu/LMMCoDrive.

翻译：为应对按需自动驾驶系统中去中心化协同调度与运动规划所面临的复杂挑战，本文提出LMMCoDrive——一种创新的协同驾驶框架，该框架利用大型多模态模型提升动态城市环境下的交通效率。该框架无缝集成调度与运动规划流程，确保协同自动驾驶车辆的高效运行。通过将CAV与乘客需求的空间关系抽象为鸟瞰图，充分释放了LMM的潜能。此外，在通过安全约束确保碰撞避免的前提下，为每辆CAV精细化优化轨迹。本文提出一种在LMM框架内基于交替方向乘子法的去中心化优化策略，以驱动CAV的图结构演化。仿真结果表明，LMM在优化CAV调度及增强各车辆去中心化协同优化过程中发挥着关键作用并产生显著影响。这标志着向实现实用、高效、安全的AMoD系统迈出了重要一步，该系统有望彻底变革城市交通。代码已发布于https://github.com/henryhcliu/LMMCoDrive。