Computerized Adaptive Testing (CAT) measures an examinee's ability while adapting to their level. Both too many questions and too many hard questions can make a test frustrating. Are there some CAT algorithms which can be proven to be theoretically better than others, and in which framework? We show that slightly extending the traditional framework yields a partial order on CAT algorithms. For uni-dimensional knowledge domains, we analyze the theoretical performance of some old and new algorithms, and we prove that none of the algorithms presented are instance optimal, conjecturing that no instance optimal can exist for the CAT problem.
翻译:计算机自适应测试(CAT)在评估受测者能力的同时,会根据其水平动态调整。题目过多或难度过高均可能使测试令人沮丧。是否存在某些CAT算法可在理论上证明优于其他算法?应在何种框架下进行论证?我们表明,对传统框架进行适度扩展,可建立CAT算法的偏序关系。针对单维知识领域,我们分析了若干新旧算法的理论性能,并证明所提出的所有算法均不具备实例最优性,进而推测CAT问题中不存在实例最优算法。