Operators experience complicated physiological and psychological states when exposed to extreme heat stress, which can impair cognitive function and decrease performance significantly, ultimately leading to severe secondary disasters. Therefore, there is an urgent need for a feasible technique to identify their abnormal states to enhance the reliability of human-cybernetics systems. With the advancement of deep learning in physiological modeling, a model for evaluating operators' performance driven by electrocardiogram (ECG) and functional near-infrared spectroscopy (fNIRS) was proposed, demonstrating high ecological validity. The model fused a convolutional neural network (CNN) backbone and a graph attention network (GAT) backbone to extract discriminative features from ECG time-frequency spectrums and fNIRS prefrontal cortex (PFC) network respectively with deeper neuroscience domain knowledge, and eventually achieved 0.90 AUC. Results supported that handcrafted features extracted by specialized neuroscience methods can alleviate overfitting. Inspired by the small-world nature of the brain network, the fNIRS PFC network was organized as an undirected graph and embedded by GAT. It is proven to perform better in information aggregation and delivery compared to a simple non-linear transformation. The model provides a potential neuroergonomics application for evaluating the human state in vital human-cybernetics systems under industry 5.0 scenarios.

翻译：操作人员在极端热应激环境下会经历复杂的生理心理状态变化，这种变化可能导致认知功能受损和作业绩效显著下降，最终引发严重的次生灾害。因此，亟需开发可行的技术手段识别操作人员的异常状态，以提升人机系统的可靠性。随着深度学习在生理建模领域的进步，本研究提出了一种基于心电图与功能性近红外光谱驱动的操作员绩效评估模型，该模型具有高生态效度。模型融合卷积神经网络主干与图注意力网络主干，分别从ECG时频谱和fNIRS前额叶皮层网络中提取具有深层次神经科学知识的判别特征，最终实现了0.90的AUC值。结果表明，采用专业神经科学方法提取的人工特征可有效缓解过拟合问题。受脑网络小世界特性启发，本研究将fNIRS前额叶皮层网络构建为无向图并通过GAT进行嵌入，实验证明该方法在信息聚合与传递方面优于简单非线性变换。该模型为工业5.0场景下关键人机系统中的人类状态评估提供了潜在的神经工效学应用方案。