The Research Excellence Framework (REF) is a periodic UK-wide assessment of the quality of published research in universities. The most recent REF was in 2014, and the next will be in 2021. The published results of REF2014 include a categorical `quality profile' for each unit of assessment (typically a university department), reporting what percentage of the unit's REF-submitted research outputs were assessed as being at each of four quality levels (labelled 4*, 3*, 2* and 1*). Also in the public domain are the original submissions made to REF2014, which include -- for each unit of assessment -- publication details of the REF-submitted research outputs. In this work, we address the question: to what extent can a REF quality profile for research outputs be attributed to the journals in which (most of) those outputs were published? The data are the published submissions and results from REF2014. The main statistical challenge comes from the fact that REF quality profiles are available only at the aggregated level of whole units of assessment: the REF panel's assessment of each individual research output is not made public. Our research question is thus an `ecological inference' problem, which demands special care in model formulation and methodology. The analysis is based on logit models in which journal-specific parameters are regularized via prior `pseudo-data'. We develop a lack-of-fit measure for the extent to which REF scores appear to depend on publication venues rather than research quality or institution-level differences. Results are presented for several research fields.

翻译：研究卓越框架（REF）是英国定期对大学已发表研究成果质量进行的全国性评估。最近一次REF于2014年进行，下一次将于2021年开展。REF2014的已公布结果包括每个评估单元（通常为一个大学院系）的分类“质量概况”，报告该单元提交REF的研究成果在各质量等级（4*、3*、2*和1*）上的百分比。公开信息中还包含原始提交给REF2014的材料——其中列出了每个评估单元所提交研究成果的发表详情。本研究探讨的问题是：研究成果的REF质量概况在多大程度上可归因于这些成果发表所在期刊？数据来自REF2014的已公布提交材料和结果。统计上的主要挑战在于REF质量概况仅以整个评估单元的汇总形式提供：REF专家组对各单项研究成果的评估并未公开。因此，我们的研究问题是一个“生态推断”问题，需要在模型构建和方法论上特别审慎。分析基于逻辑模型，其中期刊特定参数通过先验“伪数据”进行正则化。我们为REF评分在多大程度上取决于发表载体而非研究质量或机构层面差异开发了一个缺乏拟合程度的度量指标。研究结果在多个学科领域呈现。