With increasingly available computer-based or online assessments, researchers have shown keen interest in analyzing log data to improve our understanding of test takers' problem-solving processes. In this paper, we propose a multi-state survival model (MSM) to action sequence data from log files, focusing on modeling test takers' reaction times between actions, in order to investigate which factors and how they influence test takers' transition speed between actions. In particular, we focus on the effects of the occurrence and timing of key actions that differentiate correct answers from incorrect answers. We demonstrate our proposed approach with problem-solving test items from the the Programme for International Assessment of Adult Competence (PIAAC) problem-solving test items.
翻译:随着基于计算机或在线评估的日益普及,研究者对分析日志数据以加深理解受试者问题解决过程表现出浓厚兴趣。本文提出一种多状态生存模型(MSM)用于日志文件中的行为序列数据,重点建模受试者在各动作间的反应时间,旨在探究哪些因素及其如何影响受试者行为转换速度。特别地,我们关注区分正确与错误答案的关键行为的发生与时机的影响效应。通过国际成人能力评估项目(PIAAC)中的问题解决测试题,展示了所提方法的应用效果。