Biometric systems based on brain activity have been proposed as an alternative to passwords or to complement current authentication techniques. By leveraging the unique brainwave patterns of individuals, these systems offer the possibility of creating authentication solutions that are resistant to theft, hands-free, accessible, and potentially even revocable. However, despite the growing stream of research in this area, faster advance is hindered by reproducibility problems. Issues such as the lack of standard reporting schemes for performance results and system configuration, or the absence of common evaluation benchmarks, make comparability and proper assessment of different biometric solutions challenging. Further, barriers are erected to future work when, as so often, source code is not published open access. To bridge this gap, we introduce NeuroIDBench, a flexible open source tool to benchmark brainwave-based authentication models. It incorporates nine diverse datasets, implements a comprehensive set of pre-processing parameters and machine learning algorithms, enables testing under two common adversary models (known vs unknown attacker), and allows researchers to generate full performance reports and visualizations. We use NeuroIDBench to investigate the shallow classifiers and deep learning-based approaches proposed in the literature, and to test robustness across multiple sessions. We observe a 37.6% reduction in Equal Error Rate (EER) for unknown attacker scenarios (typically not tested in the literature), and we highlight the importance of session variability to brainwave authentication. All in all, our results demonstrate the viability and relevance of NeuroIDBench in streamlining fair comparisons of algorithms, thereby furthering the advancement of brainwave-based authentication through robust methodological practices.

翻译：基于脑活动的生物识别系统已被提出作为密码的替代方案或现有身份认证技术的补充。通过利用个体独特的脑波模式，这些系统能够创建既抗窃听、免手持操作、可及性高、甚至可撤销的身份认证方案。然而，尽管该领域研究日益增长，但可重复性问题阻碍了更快速的进展。如缺乏标准化的性能结果与系统配置报告方案，或缺少通用评估基准等问题，使得不同生物识别方案的比较与恰当评估变得困难。此外，当研究源码常未公开发布时，未来工作的开展也会面临障碍。为填补这一空白，我们提出NeuroIDBench——一个灵活的开源工具，用于对基于脑波的身份认证模型进行基准测试。该工具整合了九个多样化数据集，实现了全面的预处理参数集与机器学习算法，支持在两种常见对抗模型（已知攻击者与未知攻击者）下进行测试，并能生成完整的性能报告与可视化结果。我们利用NeuroIDBench系统探究了文献中提出的浅层分类器与深度学习方案，并检验了跨多个会话的鲁棒性。研究发现，在未知攻击者场景（通常未在文献中测试）下等错误率（EER）降低了37.6%，同时强调了会话变异性对脑波身份认证的重要性。总体而言，我们的研究结果证明了NeuroIDBench在简化算法公平比较中的可行性与相关性，从而通过稳健的方法论实践推动基于脑波的身份认证技术进步。