We present a novel multimodal dataset for Cognitive Load Assessment in REaltime (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset consists of four modalities, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. To map diverse levels of mental load on participants during experiments, each participant completed four nine-minutes sessions on a computer-based operator performance and mental workload task (the MATB-II software) with varying levels of complexity in one minute segments. During the experiment, participants reported their cognitive load every 10 seconds. For the dataset, we also provide benchmark binary classification results with machine learning and deep learning models on two different evaluation schemes, namely, 10-fold and leave-one-subject-out (LOSO) cross-validation. Benchmark results show that for 10-fold evaluation, the convolutional neural network (CNN) based deep learning model achieves the best classification performance with ECG, EDA, and Gaze. In contrast, for LOSO, the best performance is achieved by the deep learning model with ECG, EDA, and EEG.

翻译：我们提出了一种用于实时认知负荷评估（CLARE）的新型多模态数据集。该数据集包含来自24名参与者的生理和眼动数据，并以自我报告的认知负荷评分作为真实标签。数据集包含四种模态，即心电图（ECG）、皮肤电活动（EDA）、脑电图（EEG）和眼动追踪。为在实验中给参与者施加不同水平的心理负荷，每位参与者需在基于计算机的操作员表现与心理负荷任务（MATB-II软件）中完成四次时长为九分钟的会话，且每分钟段内任务复杂度各异。实验期间，参与者每10秒报告一次其认知负荷。对于该数据集，我们还提供了在两种不同评估方案（即10折交叉验证与留一受试者交叉验证（LOSO））下，采用机器学习与深度学习模型获得的基准二分类结果。基准结果表明，在10折评估中，基于卷积神经网络（CNN）的深度学习模型使用ECG、EDA和眼动数据取得了最佳分类性能。相比之下，在LOSO评估中，使用ECG、EDA和EEG数据的深度学习模型取得了最佳性能。