Across the social and medical sciences, researchers recognize that specifying planned research activities (i.e., 'registration') prior to the commencement of research has benefits for both the transparency and rigour of science. Despite this, evidence suggests that study registrations frequently go unexamined, minimizing their effectiveness. In a way this is no surprise: manually checking registrations against papers is labour- and time-intensive, requiring careful reading across formats and expertise across domains. The advent of AI unlocks new possibilities in facilitating this activity. We present RegCheck, a modular LLM-assisted tool designed to help researchers, reviewers, and editors from across scientific disciplines compare study registrations with their corresponding papers. Importantly, RegCheck keeps human expertise and judgement in the loop by (i) ensuring that users are the ones who determine which features should be compared, and (ii) presenting the most relevant text associated with each feature to the user, facilitating (rather than replacing) human discrepancy judgements. RegCheck also generates shareable reports with unique RegCheck IDs, enabling them to be easily shared and verified by other users. RegCheck is designed to be adaptable across scientific domains, as well as registration and publication formats. In this paper we provide an overview of the motivation, workflow, and design principles of RegCheck, and we discuss its potential as an extensible infrastructure for reproducible science with an example use case.
翻译:在社会科学与医学领域,研究者们认识到,在研究开始前预先明确规划的研究活动(即“注册”)对提升科学的透明度与严谨性具有积极作用。尽管如此,有证据表明研究注册信息往往未被充分核查,这削弱了其实际效果。这在某种程度上并不令人意外:人工核对注册内容与论文是一项劳动密集且耗时的工作,需要跨格式的仔细阅读和跨领域的专业知识。人工智能的发展为促进这一活动带来了新的可能。本文介绍RegCheck,一个模块化的、基于大语言模型辅助的工具,旨在帮助跨学科的研究者、审稿人和编辑比较研究注册与其对应论文。重要的是,RegCheck将人类专业知识与判断保留在决策闭环中,具体通过以下方式实现:(i)确保由用户自行决定需要比较哪些特征项;(ii)向用户呈现与每个特征项最相关的文本,从而辅助(而非替代)人工进行差异判断。RegCheck还可生成带有唯一RegCheck ID的可共享报告,便于其他用户轻松分享与验证。该工具设计为可适配不同科学领域以及各种注册与出版格式。本文概述了RegCheck的开发动机、工作流程与设计原则,并通过一个应用实例探讨了其作为可扩展基础设施在促进可重复科学方面的潜力。