Developing models to automatically score students' written responses to science problems is critical for science education. However, collecting and labeling sufficient student responses for training models is time and cost-consuming. Recent studies suggest that pre-trained language models can be adapted to downstream tasks without fine-tuning with prompts. However, no research has employed such a prompt approach in science education. As student responses are presented with natural language, aligning the scoring procedure as the next sentence prediction task using prompts can skip the costly fine-tuning stage. In this study, we developed a zero-shot approach to automatically score student responses via Matching Exemplars as Next Sentence Prediction (MeNSP). This approach employs no training samples. We first apply MeNSP in scoring three assessment tasks of scientific argumentation and found machine-human scoring agreements, Cohen's Kappa ranges from 0.30 to 0.57, and F1 score ranges from 0.54 to 0.81. To improve the performance, we extend our research to the few-shots setting, either randomly selecting labeled student responses or manually constructing responses to fine-tune the models. We find that one task's performance is improved with more samples, Cohen's Kappa from 0.30 to 0.38, and F1 score from 0.54 to 0.59; for the two others, scoring performance is not improved. We also find that randomly selected few-shots perform better than the human expert-crafted approach. This study suggests that MeNSP can yield referable automatic scoring for student responses while significantly reducing the cost of model training. This method can benefit low-stakes classroom assessment practices in science education. Future research should further explore the applicability of the MeNSP in different types of assessment tasks in science education and improve the model performance.

翻译：开发能够自动评阅学生对科学问题书面回答的模型对科学教育至关重要。然而，收集和标注足够的学生回答以训练模型耗时且成本高昂。近期研究表明，预训练语言模型可通过提示方式适配下游任务而无需精调。但尚无研究将此类提示方法应用于科学教育领域。由于学生回答以自然语言呈现，将评分流程对齐为基于提示的下一句预测任务可省略昂贵的精调阶段。本研究开发了一种零样本方法——通过匹配范例作为下一句预测（Matching Exemplars as Next Sentence Prediction, MeNSP）自动评阅学生回答，该方法无需训练样本。我们首先将MeNSP应用于三项科学论证评估任务的评分，发现人与机器的评分一致性指标Cohen's Kappa范围为0.30至0.57，F1分数范围为0.54至0.81。为提升性能，我们将研究拓展至少样本场景——随机选取标注学生回答或手动构建回答对模型进行精调。结果显示：一项任务的评分性能随样本量增加而提升（Cohen's Kappa从0.30升至0.38，F1分数从0.54升至0.59），但另两项任务的评分性能未获改善。我们还发现随机选取的少样本策略优于人工专家构建的方法。本研究表明MeNSP能在显著降低模型训练成本的同时获得可参考的自动评分结果，该方法可惠及科学教育中低风险课堂评估实践。未来研究应进一步探索MeNSP在科学教育不同评估任务中的适用性，并提升模型性能。