Object detection with on-board sensors (e.g., lidar, radar, and camera) play a crucial role in autonomous driving (AD), and these sensors complement each other in modalities. While crowdsensing may potentially exploit these sensors (of huge quantity) to derive more comprehensive knowledge, \textit{federated learning} (FL) appears to be the necessary tool to reach this potential: it enables autonomous vehicles (AVs) to train machine learning models without explicitly sharing raw sensory data. However, the multimodal sensors introduce various data heterogeneity across distributed AVs (e.g., label quantity skews and varied modalities), posing critical challenges to effective FL. To this end, we present AutoFed as a heterogeneity-aware FL framework to fully exploit multimodal sensory data on AVs and thus enable robust AD. Specifically, we first propose a novel model leveraging pseudo-labeling to avoid mistakenly treating unlabeled objects as the background. We also propose an autoencoder-based data imputation method to fill missing data modality (of certain AVs) with the available ones. To further reconcile the heterogeneity, we finally present a client selection mechanism exploiting the similarities among client models to improve both training stability and convergence rate. Our experiments on benchmark dataset confirm that AutoFed substantially improves over status quo approaches in both precision and recall, while demonstrating strong robustness to adverse weather conditions.

翻译：车载传感器（如激光雷达、雷达和摄像头）在自主驾驶中发挥着关键作用，且这些传感器在模态上相互补充。虽然群体感知可能利用这些（大量）传感器来获取更全面的知识，但联邦学习似乎是实现这一潜力的必要工具：它使自动驾驶车辆能够在无需显式共享原始传感数据的情况下训练机器学习模型。然而，多模态传感器在分布式自动驾驶车辆之间引入了各种数据异构性（例如，标签数量偏斜和模态变化），这对有效的联邦学习提出了严峻挑战。为此，我们提出了AutoFed作为一个异构感知的联邦学习框架，以充分利用自动驾驶车辆上的多模态传感数据，从而实现稳健的自主驾驶。具体来说，我们首先提出了一种利用伪标签的新模型，以避免错误地将未标记对象视为背景。我们还提出了一种基于自动编码器的数据插补方法，用可用的数据模态填充（某些自动驾驶车辆的）缺失数据模态。为了进一步调和异构性，我们最后提出了一种利用客户端模型之间相似性的客户端选择机制，以提高训练稳定性和收敛速度。我们在基准数据集上的实验证实，AutoFed在精度和召回率上均显著优于现有方法，同时表现出对恶劣天气条件的强鲁棒性。