Multi-sensor frameworks provide opportunities for ensemble learning and sensor fusion to make use of redundancy and supplemental information, helpful in real-world safety applications such as continuous driver state monitoring which necessitate predictions even in cases where information may be intermittently missing. We define this problem of intermittent instances of missing information (by occlusion, noise, or sensor failure) and design a learning framework around these data gaps, proposing and analyzing an imputation scheme to handle missing information. We apply these ideas to tasks in camera-based hand activity classification for robust safety during autonomous driving. We show that a late-fusion approach between parallel convolutional neural networks can outperform even the best-placed single camera model in estimating the hands' held objects and positions when validated on within-group subjects, and that our multi-camera framework performs best on average in cross-group validation, and that the fusion approach outperforms ensemble weighted majority and model combination schemes.

翻译：多传感器框架为集成学习和传感器融合提供了利用冗余与补充信息的机会，这在连续驾驶员状态监测等实际安全应用中至关重要，即使在信息可能间歇性缺失的情况下也需要做出预测。我们定义了这种由遮挡、噪声或传感器故障导致的信息间歇性缺失问题，并围绕这些数据缺口设计了一个学习框架，提出并分析了一种处理缺失信息的插补方案。我们将这些思想应用于基于摄像头的手部活动分类任务，以实现自动驾驶中的鲁棒安全。研究表明，并行卷积神经网络之间的后期融合方法在组内受试者验证中，在估计手部所持物体和位置方面甚至优于最佳的单摄像头模型；我们的多摄像头框架在跨组验证中平均表现最优，且融合方法优于集成加权多数投票和模型组合方案。