This study introduces "Survey and Questionnaire Item Embeddings Differentials" (SQuID), a novel methodological approach that enables neural network embeddings to effectively recover latent dimensions from psychometric survey items. We demonstrate that embeddings derived from large language models, when processed with SQuID, can recover the structure of human values obtained from human rater judgments on the Revised Portrait Value Questionnaire (PVQ-RR). Our experimental validation compares multiple embedding models across a number of evaluation metrics. Unlike previous approaches, SQuID successfully addresses the challenge of obtaining negative correlations between dimensions without requiring domain-specific fine-tuning. Quantitative analysis reveals that our embedding-based approach explains 55% of variance in dimension-dimension similarities compared to human data. Multidimensional scaling configurations from both types of data show fair factor congruence coefficients and largely follow the underlying theory. These results demonstrate that semantic embeddings can effectively replicate psychometric structures previously established through extensive human surveys. The approach offers substantial advantages in cost, scalability and flexibility while maintaining comparable quality to traditional methods. Our findings have significant implications for psychometrics and social science research, providing a complementary methodology that could expand the scope of human behavior and experience represented in measurement tools.
翻译:本研究提出了一种名为"调查与问卷项目嵌入差分法"(SQuID)的创新方法论,该方法使神经网络嵌入能够有效恢复心理测量调查项目的潜在维度。我们证明,当使用SQuID处理时,从大语言模型获得的嵌入能够恢复人类评分者在修订版肖像价值问卷(PVQ-RR)上判断所得的人类价值结构。我们的实验验证通过多项评估指标比较了多种嵌入模型。与先前方法不同,SQuID成功解决了在无需领域特定微调的情况下获得维度间负相关性的挑战。定量分析表明,与人类数据相比,我们基于嵌入的方法能够解释维度间相似性55%的方差。两种数据类型的多维标度配置显示出良好的因子一致性系数,并基本遵循基础理论。这些结果表明,语义嵌入能够有效复制先前通过大规模人类调查建立的心理测量结构。该方法在成本、可扩展性和灵活性方面具有显著优势,同时保持了与传统方法相当的质量。我们的发现对心理测量学和社会科学研究具有重要意义,提供了一种补充性方法论,有望扩展测量工具中表征的人类行为与经验的范围。