Camilli (2024) proposed a methodology using natural language processing (NLP) to map the relationship of a set of content standards to item specifications. This study provided evidence that NLP can be used to improve the mapping process. As part of this investigation, the nominal classifications of standards and items specifications were used to examine construct equivalence. In the current paper, we determine the strength of empirical support for the semantic distinctiveness of these classifications, which are known as "domains" for Common Core standards, and "strands" for National Assessment of Educational Progress (NAEP) item specifications. This is accomplished by separate k-means clustering for standards and specifications of their corresponding embedding vectors. We then briefly illustrate an application of these findings.
翻译:Camilli(2024)提出了一种利用自然语言处理(NLP)技术来映射内容标准集与试题规范之间关系的方法论。该研究证明了NLP可用于改进映射过程。作为本研究的一部分,我们利用标准与试题规范的名义分类来检验结构等效性。在本文中,我们通过实证方法评估了这些分类语义区分度的支持强度——这些分类在《共同核心州立标准》中称为“领域”,在全美教育进展评估(NAEP)试题规范中称为“维度”。具体实现方式是对标准及其对应试题规范的嵌入向量分别进行k均值聚类分析。最后,我们简要阐述了这些发现的应用前景。