Generative AI and large language models (LLMs) are increasingly applied to question generation and automated assessment. However, deploying LLMs in preparation for high-stakes exams requires more than prompt engineering; it demands software pipelines that systematically ground model outputs in authorised curriculum artefacts and marking guidelines issued by education authorities. This paper presents a curriculum-grounded, configurable LLM-as-Judge pipeline for question-level marking, co-developed with an industrial partner, to support exam preparation for university admission. The pipeline identifies the relevant topics, subtopics, and cognitive demand of a question, and assembles verifiable and authorised context to support LLM judgement. Curriculum intent is operationalised through concrete syllabus artefacts, including prescribed verbs and outcomes, performance band descriptors, glossary definitions, and marking-guideline principles. A staged LLM workflow is employed to first generate question-specific rubrics, capturing structured expectations of performance, and then derive and evaluate marking criteria used to allocate marks to student responses. This design improves consistency, transparency, and alignment with official marking practices. Preliminary evaluation shows that the proposed LLM-as-Judge pipeline delivers marking outcomes comparable to human tutors, while yielding justifications that are more traceable to authorised curriculum artefacts and marking standards. The pipeline has also been integrated into an online study platform, where early deployment data provide initial insights into operational usage and manual overrides.

翻译：生成式人工智能与大语言模型（LLMs）正日益应用于试题生成与自动评估领域。然而，在高风险考试备考中部署LLMs，需要的不仅是提示工程，更需构建系统化的软件管线，使模型输出严格锚定教育主管部门授权的课程文档及评分标准。本文提出一种基于课程体系、可配置的LLM-as-Judge评分管线，用于试题级评分，该管线与产业合作伙伴协同开发，旨在支持大学入学考试备考。该管线能够识别试题涉及的主题、子主题及认知需求，并整合可验证且有授权的上下文信息以支撑LLM判断。课程意图通过具体化教学大纲文档（包括规定动词与学习成果、表现等级描述符、术语表定义及评分指导原则）加以实现。采用分阶段LLM工作流程：首先生成试题专属评分细则，捕捉结构化的预期表现；随后推导并评估用于学生作答评分的关键标准。此设计提升了评分一致性、透明度以及与官方评分实践的契合度。初步评估表明，所提出的LLM-as-Judge管线在产出与人类教师可比的评分结果的同时，能提供更可追溯至授权课程文档与评分标准的判分依据。该管线已集成至在线学习平台，早期部署数据初步揭示了运行使用与人工覆写的实证特征。