Recently, in-car monitoring has emerged as a promising technology for detecting early-stage abnormal status of the driver and providing timely alerts to prevent traffic accidents. Although training models with multimodal data enhances the reliability of abnormal status detection, the scarcity of labeled data and the imbalance of class distribution impede the extraction of critical abnormal state features, significantly deteriorating training performance. Furthermore, missing modalities due to environment and hardware limitations further exacerbate the challenge of abnormal status identification. More importantly, monitoring abnormal health conditions of passengers, particularly in elderly care, is of paramount importance but remains underexplored. To address these challenges, we introduce our IC3M, an efficient camera-rotation-based multimodal framework for monitoring both driver and passengers in a car. Our IC3M comprises two key modules: an adaptive threshold pseudo-labeling strategy and a missing modality reconstruction. The former customizes pseudo-labeling thresholds for different classes based on the class distribution, generating class-balanced pseudo labels to guide model training effectively, while the latter leverages crossmodality relationships learned from limited labels to accurately recover missing modalities by distribution transferring from available modalities. Extensive experimental results demonstrate that IC3M outperforms state-of-the-art benchmarks in accuracy, precision, and recall while exhibiting superior robustness under limited labeled data and severe missing modality.

翻译：近年来，车内监测已成为一项前景广阔的技术，旨在检测驾驶员的早期异常状态并及时发出警报以预防交通事故。尽管利用多模态数据训练模型能提升异常状态检测的可靠性，但标注数据稀缺与类别分布不均衡的问题阻碍了关键异常状态特征的提取，显著降低了训练性能。此外，因环境与硬件限制导致的模态缺失进一步加剧了异常状态识别的挑战。更重要的是，监测乘客（尤其在老年照护场景中）的健康异常状态至关重要，但相关研究仍显不足。为应对这些挑战，我们提出IC3M——一种基于摄像头旋转的高效多模态框架，用于同步监测车内驾驶员与乘客。IC3M包含两个核心模块：自适应阈值伪标签策略与缺失模态重构模块。前者依据类别分布为不同类别定制伪标签阈值，生成类别均衡的伪标签以有效指导模型训练；后者则利用从有限标注中习得的跨模态关联，通过可获取模态的分布迁移精确重构缺失模态。大量实验结果表明，在标注数据有限且模态严重缺失的情况下，IC3M在准确率、精确率与召回率上均优于现有先进基准模型，同时展现出卓越的鲁棒性。