Open science initiatives have strengthened scientific integrity and accelerated research progress across many fields, but the state of their practice within transportation research remains under-investigated. Key features of open science, defined here as data and code availability, are difficult to extract due to the inherent complexity of the field. Previous work has either been limited to small-scale studies due to the labor-intensive nature of manual analysis or has relied on large-scale bibliometric approaches that sacrifice contextual richness. This paper introduces an automatic and scalable feature-extraction pipeline to measure data and code availability in transportation research. We employ Large Language Models (LLMs) for this task and validate their performance against a manually curated dataset and through an inter-rater agreement analysis. We applied this pipeline to examine 10,724 research articles published in the Transportation Research Part series of journals between 2019 and 2024. Our analysis found that only 5% of quantitative papers shared a code repository, 4% of quantitative papers shared a data repository, and about 3% of papers shared both, with trends differing across journals, topics, and geographic regions. We found no significant difference in citation counts or review duration between papers that provided data and code and those that did not, suggesting a misalignment between open science efforts and traditional academic metrics. Consequently, encouraging these practices will likely require structural interventions from journals and funding agencies to supplement the lack of direct author incentives. The pipeline developed in this study can be readily scaled to other journals, representing a critical step toward the automated measurement and monitoring of open science practices in transportation research.
翻译:开放科学倡议已在众多领域强化了科学诚信并加速了研究进展,但其在交通研究领域的实践状况仍缺乏充分调查。开放科学的关键特征——此处定义为数据与代码的可获取性——由于该领域固有的复杂性而难以提取。先前的研究或因人工分析的高强度劳动而局限于小规模研究,或依赖于牺牲上下文丰富性的大规模文献计量方法。本文提出了一种自动且可扩展的特征提取流程,用于衡量交通研究中的数据与代码可获取性。我们采用大型语言模型(LLMs)完成此任务,并通过人工标注数据集和评分者间一致性分析验证其性能。我们将此流程应用于2019年至2024年间发表在Transportation Research Part系列期刊上的10,724篇研究论文。分析发现,仅5%的定量论文共享了代码仓库,4%的定量论文共享了数据仓库,约3%的论文同时共享了二者,且趋势因期刊、主题和地理区域而异。我们发现,提供数据与代码的论文与未提供的论文在引用次数或审稿周期上均无显著差异,这表明开放科学实践与传统学术评价指标之间存在错位。因此,鼓励这些实践可能需要期刊和资助机构采取结构性干预措施,以弥补作者直接激励的不足。本研究开发的流程可轻松扩展至其他期刊,标志着向交通研究领域开放科学实践的自动化测量与监测迈出了关键一步。