The widespread usage of computer-based assessments and individualized learning platforms has resulted in an increased demand for the rapid production of high-quality items. Automated item generation (AIG), the process of using item models to generate new items with the help of computer technology, was proposed to reduce reliance on human subject experts at each step of the process. AIG has been used in test development for some time. Still, the use of machine learning algorithms has introduced the potential to improve the efficiency and effectiveness of the process greatly. The approach presented in this paper utilizes OpenAI's latest transformer-based language model, GPT-3, to generate reading passages. Existing reading passages were used in carefully engineered prompts to ensure the AI-generated text has similar content and structure to a fourth-grade reading passage. For each prompt, we generated multiple passages, the final passage was selected according to the Lexile score agreement with the original passage. In the final round, the selected passage went through a simple revision by a human editor to ensure the text was free of any grammatical and factual errors. All AI-generated passages, along with original passages were evaluated by human judges according to their coherence, appropriateness to fourth graders, and readability.
翻译:基于计算机的评估与个性化学习平台的广泛应用,使得对快速生成高质量试题的需求日益增长。自动化试题生成(AIG)技术通过借助计算机技术利用试题模型生成新试题,旨在减少对每个环节中人类学科专家的依赖。该技术在测试开发中已有应用实践,而机器学习算法的引入则进一步提升了该流程的效率与有效性。本文提出的方法采用OpenAI基于Transformer架构的最新语言模型GPT-3生成阅读段落。通过精心设计的提示工程,将既有阅读段落嵌入提示词中,确保人工智能生成文本在内容与结构上与四年级阅读段落相似。针对每个提示词,我们生成了多个备选段落,最终依据与原始段落的蓝思分级一致性选取最优段落。在最终环节,由人工编辑对选定段落进行简单修订,确保文本无语法及事实性错误。所有人工智能生成的段落及原始段落均由人类评估者根据连贯性、对四年级学生的适切性及可读性进行评价。