In this study, we evaluated the performance of the state-of-the-art sequence tagging grammar error detection and correction model (SeqTagger) using Japanese university students' writing samples. With an automatic annotation toolkit, ERRANT, we first evaluated SeqTagger's performance on error correction with human expert correction as the benchmark. Then a human-annotated approach was adopted to evaluate Seqtagger's performance in error detection using a subset of the writing dataset. Results indicated a precision of 63.66% and a recall of 20.19% for error correction in the full dataset. For the subset, after manual exclusion of irrelevant errors such as semantic and mechanical ones, the model shows an adjusted precision of 97.98% and an adjusted recall of 42.98% for error detection, indicating the model's high accuracy but also its conservativeness. Thematic analysis on errors undetected by the model revealed that determiners and articles, especially the latter, were predominant. Specifically, in terms of context-independent errors, the model occasionally overlooked basic ones and faced challenges with overly erroneous or complex structures. Meanwhile, context-dependent errors, notably those related to tense and noun number, as well as those possibly influenced by the students' first language (L1), remained particularly challenging.

翻译：本研究採用日本大學生的寫作樣本，評估了最先進的序列標註語法錯誤檢測與校正模型（SeqTagger）的效能。首先，我們以人工專家校正為基準，使用自動化標註工具ERRANT評估SeqTagger在校正錯誤方面的表現。接著，採用人工標註方法，利用寫作數據集的子集評估SeqTagger在錯誤檢測方面的表現。結果顯示，在完整數據集中，錯誤校正的精確率為63.66%，召回率為20.19%。在數據子集中，手動排除語義和機械性等不相關錯誤後，該模型在錯誤檢測方面展現出調整後精確率97.98%和調整後召回率42.98%，表明該模型具有高準確性但存在保守性。對模型未檢測到的錯誤進行主題分析發現，限定詞和冠詞（尤其是後者）佔主導地位。具體而言，上下文無關錯誤中，模型偶爾會遺漏基本錯誤，並在過多錯誤或複雜結構方面面臨挑戰。同時，上下文相關錯誤，特別是與時態和名詞數相關的錯誤，以及可能受學生母語影響的錯誤，仍然極具挑戰性。