Airwriting recognition is a task that involves identifying letters written in free space using finger movement. It is a special case of gesture recognition, where gestures correspond to letters in a specific language. Electroencephalography (EEG) is a non-invasive technique for recording brain activity and has been widely used in brain-computer interface applications. Leveraging EEG signals for airwriting recognition offers a promising alternative input method for Human-Computer Interaction. One key advantage of airwriting recognition is that users don't need to learn new gestures. By concatenating recognized letters, a wide range of words can be formed, making it applicable to a broader population. However, there has been limited research in the recognition of airwriting using EEG signals, which forms the core focus of this study. The NeuroAiR dataset comprising EEG signals recorded during writing English uppercase alphabets is first constructed. Various features are then explored in conjunction with different deep learning models to achieve accurate airwriting recognition. These features include processed EEG data, Independent Component Analysis components, source-domain-based scout time series, and spherical and head harmonic decomposition-based features. Furthermore, the impact of different EEG frequency bands on system performance is comprehensively investigated. The highest accuracy achieved in this study is 44.04% using Independent Component Analysis components and the EEGNet classification model. The results highlight the potential of EEG-based airwriting recognition as a user-friendly modality for alternative input methods in Human-Computer Interaction applications. This research sets a strong baseline for future advancements and demonstrates the viability and utility of EEG-based airwriting recognition.
翻译:空中书写识别是一项通过手指运动识别在自由空间中书写的字母的任务。它是手势识别的一个特例,其中的手势对应特定语言的字母。脑电图(EEG)是一种无创记录大脑活动的技术,已广泛用于脑机接口应用。利用EEG信号进行空中书写识别为人机交互提供了一种有前景的替代输入方法。空中书写识别的一个关键优势是用户无需学习新手势。通过拼接识别出的字母,可以形成大量单词,使其适用于更广泛的人群。然而,目前利用EEG信号进行空中书写识别的研究有限,而这正是本研究的核心。首先构建了NeuroAiR数据集,包含书写英文大写字母时的EEG信号。然后探索了多种特征与不同深度学习模型的结合,以实现准确的空中书写识别。这些特征包括处理后的EEG数据、独立成分分析(ICA)成分、基于源域的探针时间序列以及基于球面与头部谐波分解的特征。此外,全面研究了不同EEG频段对系统性能的影响。本研究使用ICA成分和EEGNet分类模型获得的最高准确率为44.04%。结果凸显了基于EEG的空中书写识别作为一种用户友好的替代输入方式在人机交互应用中的潜力。本研究为未来进展奠定了坚实基础,并展示了基于EEG的空中书写识别的可行性与实用性。