Biometric systems based on brain activity have been proposed as an alternative to passwords or to complement current authentication techniques. By leveraging the unique brainwave patterns of individuals, these systems offer the possibility of creating authentication solutions that are resistant to theft, hands-free, accessible, and potentially even revocable. However, despite the growing stream of research in this area, faster advance is hindered by reproducibility problems. Issues such as the lack of standard reporting schemes for performance results and system configuration, or the absence of common evaluation benchmarks, make comparability and proper assessment of different biometric solutions challenging. Further, barriers are erected to future work when, as so often, source code is not published open access. To bridge this gap, we introduce NeuroBench, a flexible open source tool to benchmark brainwave-based authentication models. It incorporates nine diverse datasets, implements a comprehensive set of pre-processing parameters and machine learning algorithms, enables testing under two common adversary models (known vs unknown attacker), and allows researchers to generate full performance reports and visualizations. We use NeuroBench to investigate the shallow classifiers and deep learning-based approaches proposed in the literature, and to test robustness across multiple sessions. We observe a 37.6\% reduction in Equal Error Rate (EER) for unknown attacker scenarios (typically not tested in the literature), and we highlight the importance of session variability to brainwave authentication. All in all, our results demonstrate the viability and relevance of NeuroBench in streamlining fair comparisons of algorithms, thereby furthering the advancement of brainwave-based authentication through robust methodological practices.

翻译：基于脑活动特征的生物识别系统被提出作为密码的替代方案或现有认证技术的补充。通过利用个体独特的脑电波模式，这些系统有望构造出抗窃取、免手持、普适性强甚至可撤销的认证方案。然而，尽管该领域研究日益增多，可重复性问题仍制约着其快速发展。性能报告标准与系统配置规范缺失、通用评估基准不足等问题，导致不同生物识别方案难以有效比较与客观评估。更常见的是，研究者未公开源代码，进一步阻碍了后续研究的开展。为填补这一空白，我们提出NeuroBench——一个用于脑电波身份认证模型评估的灵活开源工具。该工具整合九个公开数据集，实现全面的预处理参数与机器学习算法体系，支持两种常见攻击模型（已知攻击者与未知攻击者）测试，并允许研究人员生成完整的性能报告与可视化结果。我们利用NeuroBench评估了文献中提出的浅层分类器与深度学习模型，并测试了多会话场景下的鲁棒性。实验显示：在文献中通常未测试的未知攻击者场景下，等错误率（EER）降低37.6%；同时揭示了会话变异对脑电波身份认证的关键影响。综上，本研究成果充分验证了NeuroBench在推进算法公平比较方面的可行性与价值，通过严谨的方法论实践促进脑电波身份认证技术的发展。