Discretionary lane-change is one of the critical challenges for autonomous vehicle (AV) design due to its significant impact on traffic efficiency. Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego-vehicle, thereby suffering from limited generalization performance. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches to address the limitation of the ego vehicle-based solutions through close coordination of multiple agents. Although MARL-based approaches have shown promising results, the potential impact of lane-change decisions on the overall traffic flow of a road segment has not been fully considered. In this paper, we present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle, along with the global performance focused on the overall traffic flow of a given road segment. With a careful review of the relevant transportation literature, a novel state space is designed to integrate both the critical local traffic information pertaining to the surrounding vehicles of the ego vehicle, as well as the global traffic information obtained from a road-side unit (RSU) responsible for managing a road segment. We create a reward function to ensure that the agents make effective lane-change decisions by considering the performance of the ego vehicle and the overall improvement of traffic flow. A multi-agent deep Q-network (DQN) algorithm is designed to determine the optimal policy for each agent to effectively cooperate in performing lane-change maneuvers. LCS-TF's performance was evaluated through extensive simulations in comparison with state-of-the-art MARL models. In all aspects of traffic efficiency, driving safety, and driver comfort, the results indicate that LCS-TF exhibits superior performance.

翻译：自由变道是自动驾驶汽车设计中的关键挑战之一，因其对交通效率具有显著影响。现有智能变道方案主要聚焦于优化自车性能，从而存在泛化能力有限的缺陷。近期研究对基于多智能体强化学习的方法兴趣渐增，旨在通过多智能体的紧密协作弥补基于自车方案的局限性。尽管基于多智能体强化学习的方法已展现出可喜成果，但变道决策对道路段整体交通流的潜在影响尚未被充分考虑。本文提出一种新颖的混合多智能体强化学习智能变道系统，该系统设计用于联合优化自车的局部性能，以及针对特定道路段整体交通流的全局性能。在审慎回顾相关交通文献的基础上，我们设计了一种新颖的状态空间，同时整合了与自车周围车辆相关的关键局部交通信息，以及由管理道路段的路侧单元获取的全局交通信息。通过构建奖励函数，确保智能体在考虑自车性能与整体交通流改善的基础上做出有效变道决策。我们设计了多智能体深度Q网络算法，为每个智能体确定最优策略以实现变道操作的有效协作。通过大规模仿真实验，将LCS-TF与当前最先进的多智能体强化学习模型进行对比评估。在交通效率、行驶安全性与驾乘舒适性等所有维度上，结果表明LCS-TF均展现出更优性能。