Unidimensional factor models justify some of the most consequential summaries in science -- single scores, single ranks, and single leaderboards -- yet unidimensionality is usually assessed indirectly by fitting and evaluating models on images of the data (e.g., correlation matrices) rather than on the response matrix itself. We introduce Refactor analysis, a data-first evaluation paradigm that converts a one-factor solution into a rank-1 prediction of the original matrix by estimating both respondent- and item-side structure from dual association images. We further introduce Verifactor analysis, which evaluates the same construction under bi-cross-validated (BCV) row-column partitions for improved generalization. In simulations where the data-generating mechanism is truly rank-1 and correlational, Refactor metrics align with classical unidimensionality indices, validating the approach. However, across 200 public dichotomous datasets, traditional fit and unidimensionality measures, though highly intercorrelated, are weakly related to data recoverability, especially out of sample. This gap exposes a methodological vulnerability: excellent image-based fit can coexist with poor data-level explanatory power. Finally, treating the association measure itself as a testable hypothesis, we compare $φ$, tetrachoric, and quadrant correlation, $q^\prime$, an important reintroduction. Quadrant correlation emerges as a simple, interpretable, and remarkably robust alternative, yielding consistently stronger reconstruction and more stable behavior under sample-size variation than commonly used correlations. Together, Refactor and Verifactor shift unidimensionality assessment from "does a one-factor model fit the correlation matrix?" to the question that matters for measurement and benchmarking: does a one-factor dependence structure recover and generalize the observed responses?
翻译:单维因子模型支撑着科学中一些最具影响力的总结——单一分数、单一排名和单一排行榜——然而,单维性通常是通过在数据影像(如相关矩阵)上拟合和评估模型来间接评估,而非直接在响应矩阵本身上进行。我们引入"重构分析"(Refactor analysis),这是一种数据优先的评估范式,它将单因子解转化为原始矩阵的秩-1预测,通过从双关联影像中同时估计响应者和项目侧结构来实现。我们进一步引入"验证分析"(Verifactor analysis),在双交叉验证的行列分区下评估同一构造,以提高泛化能力。在数据生成机制真正为秩-1且相关性的模拟中,重构指标与经典单维性指数保持一致,验证了该方法。然而,在200个公开二值数据集中,传统拟合与单维性度量虽然高度互相关,但与数据可恢复性(尤其是样本外)关联较弱。这一差距暴露了方法论上的脆弱性:基于影像的出色拟合可能与低水平的数据层面解释力共存。最后,将关联度量本身作为可检验的假设,我们比较了$φ$、四分相关和象限相关$q^\prime$——这是一种重要的重新引入。象限相关作为一种简单、可解释且极其稳健的替代方案脱颖而出,在样本量变化下比常用相关性产生持续更强的重构和更稳定的行为。综上所述,重构分析与验证分析将单维性评估从"单因子模型是否拟合相关矩阵?"转向对测量和基准测试真正重要的问题:单因子依赖结构是否能够恢复并泛化观测到的响应?