This paper presents an experiment of automatically scoring handwritten descriptive answers in the trial tests for the new Japanese university entrance examination, which were made for about 120,000 examinees in 2017 and 2018. There are about 400,000 answers with more than 20 million characters. Although all answers have been scored by human examiners, handwritten characters are not labeled. We present our attempt to adapt deep neural network-based handwriting recognizers trained on a labeled handwriting dataset into this unlabeled answer set. Our proposed method combines different training strategies, ensembles multiple recognizers, and uses a language model built from a large general corpus to avoid overfitting into specific data. In our experiment, the proposed method records character accuracy of over 97% using about 2,000 verified labeled answers that account for less than 0.5% of the dataset. Then, the recognized answers are fed into a pre-trained automatic scoring system based on the BERT model without correcting misrecognized characters and providing rubric annotations. The automatic scoring system achieves from 0.84 to 0.98 of Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents an acceptable similarity of scoring between the automatic scoring system and the human examiners. These results are promising for further research on end-to-end automatic scoring of descriptive answers.

翻译：本文介绍了一项实验，针对2017年和2018年约12万名考生参加的新型日本大学入学考试模拟测试中的手写描述性答案进行自动评分。实验涉及约40万份答案，总字符数超过2000万。尽管所有答案均已由人工评分员完成评分，但手写字符并未标注。我们提出将基于深度神经网络的手写识别模型（该模型已在标注手写数据集上训练）适配至这一未标注答案集的方法。所提方法结合多种训练策略、集成多个识别模型，并利用基于大规模通用语料库构建的语言模型以避免对特定数据的过拟合。实验表明，该方法在仅使用约2000个经核查的标注答案（占数据集的不到0.5%）时，字符准确率超过97%。随后，将识别后的答案输入基于BERT模型的预训练自动评分系统，无需纠正误识别字符或提供评分标准注释。该自动评分系统在二次加权卡帕系数（QWK）上达到0.84至0.98。QWK超过0.8表明自动评分系统与人工评分员之间具有可接受的评分一致性。这些结果为进一步开展端到端描述性答案自动评分研究提供了前景广阔的基础。