Object detection with on-board sensors (e.g., lidar, radar, and camera) play a crucial role in autonomous driving (AD), and these sensors complement each other in modalities. While crowdsensing may potentially exploit these sensors (of huge quantity) to derive more comprehensive knowledge, \textit{federated learning} (FL) appears to be the necessary tool to reach this potential: it enables autonomous vehicles (AVs) to train machine learning models without explicitly sharing raw sensory data. However, the multimodal sensors introduce various data heterogeneity across distributed AVs (e.g., label quantity skews and varied modalities), posing critical challenges to effective FL. To this end, we present AutoFed as a heterogeneity-aware FL framework to fully exploit multimodal sensory data on AVs and thus enable robust AD. Specifically, we first propose a novel model leveraging pseudo-labeling to avoid mistakenly treating unlabeled objects as the background. We also propose an autoencoder-based data imputation method to fill missing data modality (of certain AVs) with the available ones. To further reconcile the heterogeneity, we finally present a client selection mechanism exploiting the similarities among client models to improve both training stability and convergence rate. Our experiments on benchmark dataset confirm that AutoFed substantially improves over status quo approaches in both precision and recall, while demonstrating strong robustness to adverse weather conditions.

翻译：车载传感器（如激光雷达、雷达和摄像头）的目标检测在自动驾驶中起着关键作用，且这些传感器在模态上相互补充。尽管群智感知可能利用这些大量传感器获取更全面的知识，但联邦学习（FL）似乎是实现这一潜力的必要工具：它使自动驾驶车辆能够在无需显式共享原始传感数据的情况下训练机器学习模型。然而，多模态传感器在分布式自动驾驶车辆间引入了多种数据异构性（例如标签数量偏差和模态差异），对有效联邦学习构成了严峻挑战。为此，我们提出AutoFed作为异构感知的联邦学习框架，以充分利用自动驾驶车辆上的多模态传感数据，从而实现鲁棒的自动驾驶。具体而言，我们首先提出一种利用伪标签的新模型，避免将未标记目标误判为背景。其次，我们提出一种基于自动编码器的数据插补方法，利用可用模态填补（特定自动驾驶车辆的）缺失数据模态。为进一步协调异构性，我们最后提出一种利用客户端模型相似性的客户端选择机制，以提高训练稳定性和收敛速度。在基准数据集上的实验表明，AutoFed在精确率和召回率上均显著优于现有方法，并在恶劣天气条件下展现出强鲁棒性。