Computerized Adaptive Testing(CAT) refers to an online system that adaptively selects the best-suited question for students with various abilities based on their historical response records. Most CAT methods only focus on the quality objective of predicting the student ability accurately, but neglect concept diversity or question exposure control, which are important considerations in ensuring the performance and validity of CAT. Besides, the students' response records contain valuable relational information between questions and knowledge concepts. The previous methods ignore this relational information, resulting in the selection of sub-optimal test questions. To address these challenges, we propose a Graph-Enhanced Multi-Objective method for CAT (GMOCAT). Firstly, three objectives, namely quality, diversity and novelty, are introduced into the Scalarized Multi-Objective Reinforcement Learning framework of CAT, which respectively correspond to improving the prediction accuracy, increasing the concept diversity and reducing the question exposure. We use an Actor-Critic Recommender to select questions and optimize three objectives simultaneously by the scalarization function. Secondly, we utilize the graph neural network to learn relation-aware embeddings of questions and concepts. These embeddings are able to aggregate neighborhood information in the relation graphs between questions and concepts. We conduct experiments on three real-world educational datasets, and show that GMOCAT not only outperforms the state-of-the-art methods in the ability prediction, but also achieve superior performance in improving the concept diversity and alleviating the question exposure. Our code is available at https://github.com/justarter/GMOCAT.
翻译:计算机化自适应测试(CAT)指的是一个在线系统,能够根据学生历史作答记录,自适应地为不同能力水平的学生选择最合适的题目。大多数CAT方法仅关注准确预测学生能力的质量目标,却忽视了概念多样性和题目曝光控制这两个对保障CAT性能与效度至关重要的因素。此外,学生作答记录包含了题目与知识概念之间的有价值关系信息,以往方法忽略了这一关系信息,导致选出次优的测试题目。为解决这些挑战,我们提出了一种图增强的多目标CAT方法(GMOCAT)。首先,我们将质量、多样性和新颖性三个目标引入CAT的标量化多目标强化学习框架中,分别对应提升预测精度、增加概念多样性和降低题目曝光度。我们采用Actor-Critic推荐器来选择题目,并通过标量化函数同时优化三个目标。其次,我们利用图神经网络学习题目与概念的关系感知嵌入表示,这些嵌入能够聚合题目与概念关系图中的邻域信息。我们在三个真实教育数据集上进行了实验,结果表明GMOCAT不仅在能力预测上优于现有最先进方法,在提升概念多样性和缓解题目曝光问题方面也取得了更优性能。我们的代码已开源在https://github.com/justarter/GMOCAT。