Thanks to the rapid growth in wearable technologies and recent advancement in machine learning and signal processing, monitoring complex human contexts becomes feasible, paving the way to develop human-in-the-loop IoT systems that naturally evolve to adapt to the human and environment state autonomously. Nevertheless, a central challenge in designing many of these IoT systems arises from the requirement to infer the human mental state, such as intention, stress, cognition load, or learning ability. While different human contexts can be inferred from the fusion of different sensor modalities that can correlate to a particular mental state, the human brain provides a richer sensor modality that gives us more insights into the required human context. This paper proposes ERUDITE, a human-in-the-loop IoT system for the learning environment that exploits recent wearable neurotechnology to decode brain signals. Through insights from concept learning theory, ERUDITE can infer the human state of learning and understand when human learning increases or declines. By quantifying human learning as an input sensory signal, ERUDITE can provide adequate personalized feedback to humans in a learning environment to enhance their learning experience. ERUDITE is evaluated across $15$ participants and showed that by using the brain signals as a sensor modality to infer the human learning state and providing personalized adaptation to the learning environment, the participants' learning performance increased on average by $26\%$. Furthermore, we showed that ERUDITE can be deployed on an edge-based prototype to evaluate its practicality and scalability.
翻译:得益于可穿戴技术的迅速发展以及机器学习与信号处理的最新进展,监测复杂的人体情境已成为可能,这为开发能够自主适应人类与环境的闭环人机交互物联网系统铺平了道路。然而,设计此类物联网系统的一个核心挑战在于需要推断人类的心理状态(如意图、压力、认知负荷或学习能力)。尽管不同传感器模态的融合可推断特定心理状态对应的人体情境,但人脑作为一种更丰富的传感器模态,能为所需的人体情境提供更深刻的见解。本文提出ERUDITE系统——一种面向学习环境的人机协同物联网系统,利用新兴的可穿戴神经技术解码脑信号。通过概念学习理论的启示,ERUDITE能够推断人类的学习状态,并理解人类学习能力提升或下降的时机。通过将人类学习量化为输入传感信号,ERUDITE可在学习环境中为人类提供充分的个性化反馈,以增强其学习体验。通过对15名参与者的实验评估,结果表明:当使用脑信号作为传感器模态推断人类学习状态,并对学习环境提供个性化自适应调整时,参与者的学习效果平均提升了26%。此外,我们展示了ERUDITE可部署于边缘原型系统,以验证其实用性与可扩展性。