Driver monitoring systems (DMS) are a key component of vehicular safety and essential for the transition from semiautonomous to fully autonomous driving. A key task for DMS is to ascertain the cognitive state of a driver and to determine their level of tiredness. Neuromorphic vision systems, based on event camera technology, provide advanced sensing of facial characteristics, in particular the behavior of a driver's eyes. This research explores the potential to extend neuromorphic sensing techniques to analyze the entire facial region, detecting yawning behaviors that give a complimentary indicator of tiredness. A neuromorphic dataset is constructed from 952 video clips (481 yawns, 471 not-yawns) captured with an RGB color camera, with 37 subjects. A total of 95200 neuromorphic image frames are generated from this video data using a video-to-event converter. From these data 21 subjects were selected to provide a training dataset, 8 subjects were used for validation data, and the remaining 8 subjects were reserved for an "unseen" test dataset. An additional 12300 frames were generated from event simulations of a public dataset to test against other methods. A CNN with self-attention and a recurrent head was designed, trained, and tested with these data. Respective precision and recall scores of 95.9 percent and 94.7 percent were achieved on our test set, and 89.9 percent and 91 percent on the simulated public test set, demonstrating the feasibility to add yawn detection as a sensing component of a neuromorphic DMS.

翻译：驾驶员监控系统（DMS）是车辆安全的关键组成部分，也是从半自动驾驶向全自动驾驶过渡的必要条件。DMS的核心任务之一是评估驾驶员的认知状态并确定其疲劳程度。基于事件相机技术的神经形态视觉系统，能够实现对面部特征（尤其是驾驶员眼部行为）的先进感知。本研究探索将神经形态传感技术扩展至全脸区域分析的可能性，通过检测哈欠行为提供疲劳状态的互补性指标。实验构建了一个包含37名受试者、由RGB彩色相机采集的952段视频片段（481段哈欠视频，471段非哈欠视频）的神经形态数据集。通过视频-事件转换器，从这些视频数据中生成了总计95,200帧神经形态图像帧。从中选取21名受试者的数据作为训练集，8名受试者的数据作为验证集，剩余8名受试者的数据作为"未见"测试集。此外，基于公开数据集的事件仿真生成了12,300帧用于对比测试。本文设计并训练了一种融合自注意力机制与循环神经网络头的卷积神经网络（CNN），并利用上述数据进行测试。在自建测试集上分别实现了95.9%的精确率与94.7%的召回率，在仿真公开测试集上达到89.9%的精确率与91%的召回率，验证了将哈欠检测作为神经形态DMS感知组件的可行性。