Visual speaker recognition based on lip motion offers a silent, hands-free, and behavior-driven biometric solution that remains effective even when acoustic cues are unavailable. Compared to traditional methods that rely heavily on appearance-dependent representations, lip motion encodes subject-specific behavioral dynamics driven by consistent articulation patterns and muscle coordination, offering inherent stability across environmental changes. However, capturing these robust, fine-grained dynamics is challenging for conventional frame-based cameras due to motion blur and low dynamic range. To exploit the intrinsic stability of lip motion and address these sensing limitations, we propose NeuroLip, an event-based framework that captures fine-grained lip dynamics under a strict yet practical cross-scene protocol: training is performed under a single controlled condition, while recognition must generalize to unseen viewing and lighting conditions. NeuroLip features a 1) Temporal-aware Voxel Encoding module with adaptive event weighting, 2) Structure-aware Spatial Enhancer that amplifies discriminative behavioral patterns by suppressing noise while preserving vertically structured motion information, and 3) Polarity Consistency Regularization mechanism to retain motion-direction cues encoded in event polarities. To facilitate systematic evaluation, we introduce DVSpeaker, a comprehensive event-based lip-motion dataset comprising 50 subjects recorded under four distinct viewpoint and illumination scenarios. Extensive experiments demonstrate that NeuroLip achieves near-perfect matched-scene accuracy and robust cross-scene generalization, attaining over 71% accuracy on unseen viewpoints and nearly 76% under low-light conditions, outperforming representative existing methods by at least 8.54%. The dataset and code are publicly available at https://github.com/JiuZeongit/NeuroLip.
翻译:基于唇动的视觉说话人识别提供了一种无声、免接触且行为驱动的生物特征解决方案,即使在声学线索缺失时仍能保持有效。相较于依赖外观表征的传统方法,唇动编码了由一致发音模式和肌肉协调驱动的主体特定行为动态,在环境变化中表现出内在稳定性。然而,传统帧式相机因运动模糊和低动态范围难以捕获这些鲁棒的细粒度动态。为利用唇动的内在稳定性并克服传感限制,我们提出NeuroLip这一基于事件的框架,在严格且实用的跨场景协议下捕获细粒度唇动动态:在单一受控条件下训练,而识别需泛化至未见视角与光照条件。NeuroLip包含:1)带自适应事件加权的时序感知体素编码模块,2)通过抑制噪声同时保留垂直结构运动信息来增强判别性行为模式的结构感知空间增强器,3)极性一致性正则化机制以保留事件极性编码的运动方向线索。为促进系统评估,我们构建DVSpeaker这一综合事件型唇动数据集,包含50名受试者在四种视角与光照场景下的记录。大量实验表明,NeuroLip在匹配场景下接近完美准确率,并展现鲁棒的跨场景泛化能力:在未见视角下准确率超71%,在弱光条件下接近76%,至少以8.54%的绝对优势超越现有代表性方法。数据集与代码已公开于https://github.com/JiuZeongit/NeuroLip。