Driver activity classification is crucial for ensuring road safety, with applications ranging from driver assistance systems to autonomous vehicle control transitions. In this paper, we present a novel approach leveraging generalizable representations from vision-language models for driver activity classification. Our method employs a Semantic Representation Late Fusion Neural Network (SRLF-Net) to process synchronized video frames from multiple perspectives. Each frame is encoded using a pretrained vision-language encoder, and the resulting embeddings are fused to generate class probability predictions. By leveraging contrastively-learned vision-language representations, our approach achieves robust performance across diverse driver activities. We evaluate our method on the Naturalistic Driving Action Recognition Dataset, demonstrating strong accuracy across many classes. Our results suggest that vision-language representations offer a promising avenue for driver monitoring systems, providing both accuracy and interpretability through natural language descriptors.

翻译：驾驶员行为分类对保障道路安全至关重要，其应用涵盖驾驶员辅助系统至自动驾驶车辆控制切换等领域。本文提出一种利用视觉-语言模型通用表征进行驾驶员行为分类的新方法。该方法采用语义表征后期融合神经网络（SRLF-Net）处理来自多个视角的同步视频帧。通过预训练的视觉-语言编码器对每一帧进行编码，并将生成的嵌入特征进行融合以产生类别概率预测。借助对比学习的视觉-语言表征，该方法在各类驾驶员行为中均表现出稳健性能。我们在自然驾驶动作识别数据集上验证了该方法的有效性，并在多个类别上展现出高准确率。研究结果表明，视觉-语言表征为驾驶员监控系统提供了兼具准确性与可解释性的可行路径——这得益于自然语言描述符的支撑。