Thanks to the rapid growth in wearable technologies and recent advancement in machine learning and signal processing, monitoring complex human contexts becomes feasible, paving the way to develop human-in-the-loop IoT systems that naturally evolve to adapt to the human and environment state autonomously. Nevertheless, a central challenge in designing many of these IoT systems arises from the requirement to infer the human mental state, such as intention, stress, cognition load, or learning ability. While different human contexts can be inferred from the fusion of different sensor modalities that can correlate to a particular mental state, the human brain provides a richer sensor modality that gives us more insights into the required human context. This paper proposes ERUDITE, a human-in-the-loop IoT system for the learning environment that exploits recent wearable neurotechnology to decode brain signals. Through insights from concept learning theory, ERUDITE can infer the human state of learning and understand when human learning increases or declines. By quantifying human learning as an input sensory signal, ERUDITE can provide adequate personalized feedback to humans in a learning environment to enhance their learning experience. ERUDITE is evaluated across $15$ participants and showed that by using the brain signals as a sensor modality to infer the human learning state and providing personalized adaptation to the learning environment, the participants' learning performance increased on average by $26\%$. Furthermore, we showed that ERUDITE can be deployed on an edge-based prototype to evaluate its practicality and scalability.
翻译:摘要:得益于可穿戴技术的快速进步以及机器学习和信号处理领域的最新突破,监测复杂的人类情境已成为可能,这为开发能够自主适应人类与环境状态的闭环物联网系统铺平了道路。然而,设计这类物联网系统的核心挑战在于需要推断人类心理状态,例如意图、压力、认知负荷或学习能力。虽然不同的人类情境可通过融合多种传感器模态进行推断(这些模态与特定心理状态相关),但人类大脑作为一种更丰富的传感器模态,能为所需的人类情境提供更深入的洞察。本文提出ERUDITE——一种面向学习环境的闭环物联网系统,它利用最新的可穿戴神经技术来解码脑信号。通过结合概念学习理论的见解,ERUDITE能够推断人类的学习状态,并理解人类学习能力何时增强或减弱。通过将人类学习量化为输入感知信号,ERUDITE可在学习环境中向人类提供充分的个性化反馈,以提升其学习体验。ERUDITE在15名参与者中进行了评估,结果表明:通过将脑信号作为传感器模态来推断人类学习状态,并对学习环境进行个性化自适应调整,参与者的学习表现平均提升了26%。此外,我们展示了ERUDITE可部署于边缘计算原型上,以评估其实用性与可扩展性。