The integration of artificial intelligence (AI) into healthcare has advanced significantly, yet affect recognition remains a major challenge, particularly in AI-assisted interventions such as Computerized Cognitive Training (CCT). The THERADIA-WoZ corpus was developed to enable multimodal affect recognition in the context of AI-driven CCT, focusing on an older adult population. This study extends the corpus by introducing a dataset collected from young adults, allowing direct comparison of affect recognition models across age groups. Our objective was to assess whether multimodal models based on dimensions borrowed from appraisal theories outperform those based on categorical labels and to evaluate their generalisation power across age corpora. After comparing both corpora, models were trained and tested using within-corpus, cross-corpus, and mixed-corpus evaluation. Results revealed that appraisal dimensions consistently outperformed categorical labels across all conditions, demonstrating greater predictive accuracy and stability. Notably, categorical labels failed to generalise across age corpora, as performance dropped to chance levels in cross-corpus evaluation. In contrast, appraisal dimensions maintained predictive performance above chance, reinforcing their robustness for cross-age affect recognition. Furthermore, training on both corpora did not improve generalisation beyond within-corpus training. The findings support the theoretical and practical advantages of appraisal dimensions over categorical labels in affective computing. They also highlight the importance of multimodal fusion and deep learning representations for emotion modeling. To facilitate future research, we provide an API for researchers interested in time-continuous emotion prediction, offering valuable tools for behavioral sciences to enhance the measurement of emotional states in various experimental settings.
翻译:人工智能在医疗健康领域的整合取得了显著进展,但情感识别仍是一项重大挑战,尤其是在计算机化认知训练等人工智能辅助干预中。THERADIA-WoZ语料库旨在支持人工智能驱动的计算机化认知训练背景下的多模态情感识别,重点关注老年群体。本研究通过引入从年轻成年人中收集的数据集扩展了该语料库,从而能够直接比较不同年龄组的情感识别模型。我们的目标是评估基于评估理论维度的多模态模型是否优于基于分类标签的模型,并评估它们在跨年龄语料库中的泛化能力。在比较两个语料库后,模型通过语料库内评估、跨语料库评估和混合语料库评估进行训练和测试。结果表明,在所有条件下,评估维度始终优于分类标签,展现出更高的预测准确性和稳定性。值得注意的是,分类标签无法在跨年龄语料库中泛化,因为在跨语料库评估中其性能降至随机水平。相反,评估维度保持高于随机水平的预测性能,进一步证明了其在跨年龄情感识别中的稳健性。此外,在双语料库上训练并未比语料库内训练改善泛化效果。研究结果支持了评估维度相对于分类标签在情感计算中的理论和实践优势,同时强调了多模态融合和深度学习表征在情感建模中的重要性。为促进未来研究,我们为对时间连续情感预测感兴趣的研究者提供了应用程序接口,为行为科学提供了宝贵工具,以增强不同实验设置中情绪状态的测量能力。