This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, AACR, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, AACR, and BERT, respectively, for five scoring aspects (p<0.001). The overall HNN's perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model's accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to score students' science writing automatically.
翻译:本研究探索了多视角混合神经网络(HNN)在科学教育中依据分析量规对学生回答进行评分的有效性。我们将HNN模型与四种机器学习方法(BERT、AACR、朴素贝叶斯和逻辑回归)的准确性进行了比较。结果显示,在五个评分维度上(p<0.001),HNN的准确率分别比朴素贝叶斯、逻辑回归、AACR和BERT高出8%、3%、1%和0.12%。HNN的整体感知准确率(均值=96.23%,标准差=1.45%)与计算成本高昂(训练和推理)的BERT模型准确率(均值=96.12%,标准差=1.52%)相当。我们还观察到,HNN在训练和推理效率上比BERT快2倍,且与轻量级但准确率较低的朴素贝叶斯模型效率相当。本研究证实了使用HNN自动评分学生科学写作的准确性和效率。