Object detection with on-board sensors (e.g., lidar, radar, and camera) play a crucial role in autonomous driving (AD), and these sensors complement each other in modalities. While crowdsensing may potentially exploit these sensors (of huge quantity) to derive more comprehensive knowledge, \textit{federated learning} (FL) appears to be the necessary tool to reach this potential: it enables autonomous vehicles (AVs) to train machine learning models without explicitly sharing raw sensory data. However, the multimodal sensors introduce various data heterogeneity across distributed AVs (e.g., label quantity skews and varied modalities), posing critical challenges to effective FL. To this end, we present AutoFed as a heterogeneity-aware FL framework to fully exploit multimodal sensory data on AVs and thus enable robust AD. Specifically, we first propose a novel model leveraging pseudo-labeling to avoid mistakenly treating unlabeled objects as the background. We also propose an autoencoder-based data imputation method to fill missing data modality (of certain AVs) with the available ones. To further reconcile the heterogeneity, we finally present a client selection mechanism exploiting the similarities among client models to improve both training stability and convergence rate. Our experiments on benchmark dataset confirm that AutoFed substantially improves over status quo approaches in both precision and recall, while demonstrating strong robustness to adverse weather conditions.

翻译：搭载车载传感器（如激光雷达、雷达和摄像头）的目标检测在自动驾驶中扮演关键角色，且这些传感器在模态上相互补充。虽然群智感知可能利用这些（海量）传感器获取更全面的知识，但联邦学习似乎是实现这一潜力的必要工具：它使自动驾驶车辆能够在无需显式共享原始传感数据的情况下训练机器学习模型。然而，多模态传感器在分布式自动驾驶车辆中引入了多种数据异构性（例如标签数量偏斜和模态差异），这对有效的联邦学习构成了严峻挑战。为此，我们提出AutoFed作为异构感知联邦学习框架，以充分利用自动驾驶车辆上的多模态传感数据，从而实现稳健的自动驾驶。具体而言，我们首先提出一种利用伪标签的新型模型，以避免错误地将未标记对象视为背景。我们还提出了一种基于自编码器的数据填补方法，用可用模态填补（某些车辆的）缺失数据模态。为进一步协调异构性，我们最终提出一种客户选择机制，利用客户模型间的相似性来提升训练稳定性和收敛速度。在基准数据集上的实验证实，AutoFed在精确率和召回率上均显著优于现有方法，同时展现出对恶劣天气条件的强大鲁棒性。