This paper presents VisioPhysioENet, a novel multimodal system that leverages visual cues and physiological signals to detect learner engagement. It employs a two-level approach for visual feature extraction using the Dlib library for facial landmark extraction and the OpenCV library for further estimations. This is complemented by extracting physiological signals using the plane-orthogonal-to-skin method to assess cardiovascular activity. These features are integrated using advanced machine learning classifiers, enhancing the detection of various engagement levels. We rigorously evaluate VisioPhysioENet on the DAiSEE dataset, where it achieves an accuracy of 63.09%, demonstrating a superior ability to discern various levels of engagement compared to existing methodologies. The proposed system's code can be accessed at https://github.com/MIntelligence-Group/VisioPhysioENet.

翻译：本文提出VisioPhysioENet，一种利用视觉线索与生理信号检测学习者投入度的新型多模态系统。该系统采用两级视觉特征提取方法：首先使用Dlib库进行面部关键点提取，继而通过OpenCV库进行进一步估算。同时，采用皮肤正交平面法提取生理信号以评估心血管活动。通过先进的机器学习分类器融合这些特征，从而提升对不同投入水平的检测能力。我们在DAiSEE数据集上对VisioPhysioENet进行了严格评估，其准确率达到63.09%，相较于现有方法展现出更优异的多元投入水平辨别能力。该系统的代码可通过https://github.com/MIntelligence-Group/VisioPhysioENet 获取。