Deep learning-based Autonomous Driving (AD) models often exhibit poor generalization due to data heterogeneity in an ever domain-shifting environment. While Federated Learning (FL) could improve the generalization of an AD model (known as FedAD system), conventional models often struggle with under-fitting as the amount of accumulated training data progressively increases. To address this issue, instead of conventional small models, employing Large Vision Models (LVMs) in FedAD is a viable option for better learning of representations from a vast volume of data. However, implementing LVMs in FedAD introduces three challenges: (I) the extremely high communication overheads associated with transmitting LVMs between participating vehicles and a central server; (II) lack of computing resource to deploy LVMs on each vehicle; (III) the performance drop due to LVM focusing on shared features but overlooking local vehicle characteristics. To overcome these challenges, we propose pFedLVM, a LVM-Driven, Latent Feature-Based Personalized Federated Learning framework. In this approach, the LVM is deployed only on central server, which effectively alleviates the computational burden on individual vehicles. Furthermore, the exchange between central server and vehicles are the learned features rather than the LVM parameters, which significantly reduces communication overhead. In addition, we utilize both shared features from all participating vehicles and individual characteristics from each vehicle to establish a personalized learning mechanism. This enables each vehicle's model to learn features from others while preserving its personalized characteristics, thereby outperforming globally shared models trained in general FL. Extensive experiments demonstrate that pFedLVM outperforms the existing state-of-the-art approaches.

翻译：基于深度学习的自动驾驶模型在持续域偏移的环境中常因数据异构性而表现出较差的泛化能力。虽然联邦学习能够提升自动驾驶模型的泛化性能（即联邦自动驾驶系统），但随着累积训练数据量的逐步增加，传统模型往往因欠拟合而受限。为解决此问题，在联邦自动驾驶中采用大视觉模型替代传统小模型，是从海量数据中学习表征的有效途径。然而，在联邦自动驾驶中部署大视觉模型面临三大挑战：（I）参与车辆与中央服务器间传输大视觉模型产生的极高通信开销；（II）单车缺乏部署大视觉模型的计算资源；（III）大视觉模型聚焦共享特征而忽略本地车辆特性导致的性能下降。为克服这些挑战，我们提出pFedLVM——一种大视觉模型驱动、基于潜在特征的个性化联邦学习框架。该框架仅将大视觉模型部署于中央服务器，有效减轻了单车的计算负担。此外，中央服务器与车辆间交换的是学习所得特征而非大视觉模型参数，从而显著降低通信开销。同时，我们综合利用所有参与车辆的共享特征与各车辆的个体特性，构建个性化学习机制。这使得每辆车的模型既能学习其他车辆的特征，又能保留其个性化特性，从而超越通用联邦学习中训练的全局共享模型。大量实验表明，pFedLVM在性能上优于现有最先进方法。