Event-based cameras are bio-inspired sensors that asynchronously capture pixel intensity changes with microsecond latency, high temporal resolution, and high dynamic range, providing information on the spatiotemporal dynamics of a scene. We propose evTransFER, a transfer learning-based framework for facial expression recognition using event-based cameras. The main contribution is a feature extractor designed to encode facial spatiotemporal dynamics, built by training an adversarial generative method on facial reconstruction and transferring the encoder weights to the facial expression recognition system. We demonstrate that the proposed transfer learning method improves facial expression recognition compared to training a network from scratch. We propose an architecture that incorporates an LSTM to capture longer-term facial expression dynamics and introduces a new event-based representation called TIE. We evaluated the framework using both the synthetic event-based facial expression database e-CK+ and the real neuromorphic dataset NEFER. On e-CK+, evTransFER achieved a recognition rate of 93.6\%, surpassing state-of-the-art methods. For NEFER, which comprises event sequence with real sensor noise and sparse activity, the proposed transfer learning strategy achieved an accuracy of up to 76.7\%. In both datasets, the outcomes surpassed current methodologies and exceeded results when compared with models trained from scratch.

翻译：事件相机是一种仿生传感器，能够以微秒级延迟、高时间分辨率和高动态范围异步捕获像素强度变化，从而提供场景的时空动态信息。本文提出evTransFER，一个基于迁移学习的框架，用于利用事件相机进行面部表情识别。其主要贡献在于设计了一个用于编码面部时空动态特征的特征提取器，该提取器通过在面部重建任务上训练对抗生成方法构建，并将其编码器权重迁移至面部表情识别系统。实验证明，与从头开始训练网络相比，所提出的迁移学习方法提升了面部表情识别性能。我们提出了一种结合LSTM以捕捉长期面部表情动态的架构，并引入了一种称为TIE的新型事件表示方法。该框架在合成事件面部表情数据库e-CK+和真实神经形态数据集NEFER上进行了评估。在e-CK+上，evTransFER取得了93.6%的识别率，超越了现有先进方法。对于包含真实传感器噪声和稀疏活动的NEFER事件序列，所提出的迁移学习策略达到了最高76.7%的准确率。在两个数据集中，该方法均优于当前主流技术，且显著超越了从头训练模型的性能。