The goal of screening prioritisation in systematic reviews is to identify relevant documents with high recall and rank them in early positions for review. This saves reviewing effort if paired with a stopping criterion, and speeds up review completion if performed alongside downstream tasks. Recent studies have shown that neural models have good potential on this task, but their time-consuming fine-tuning and inference discourage their widespread use for screening prioritisation. In this paper, we propose an alternative approach that still relies on neural models, but leverages dense representations and relevance feedback to enhance screening prioritisation, without the need for costly model fine-tuning and inference. This method exploits continuous relevance feedback from reviewers during document screening to efficiently update the dense query representation, which is then applied to rank the remaining documents to be screened. We evaluate this approach across the CLEF TAR datasets for this task. Results suggest that the investigated dense query-driven approach is more efficient than directly using neural models and shows promising effectiveness compared to previous methods developed on the considered datasets. Our code is available at https://github.com/ielab/dense-screening-feedback.
翻译:系统综述中筛选优先级排序的目标是以高召回率识别相关文献,并将其排列在早期审查位置。若与停止准则结合使用,可节省审查工作量;若与下游任务并行执行,则可加速综述完成。近期研究表明,神经网络模型在此任务上具有良好潜力,但其耗时的微调与推理过程阻碍了其在筛选优先级排序中的广泛应用。本文提出一种替代方案,该方法仍依赖神经网络模型,但利用密集表示与相关性反馈增强筛选优先级排序,无需昂贵的模型微调与推理过程。该方法通过文献筛选过程中评审者提供的连续相关性反馈,高效更新密集查询表示,进而对剩余待筛选文献进行排序。我们在CLEF TAR数据集上对此方法进行评估。结果表明:所研究的密集查询驱动方法比直接使用神经网络模型更高效,与现有数据集上已开发方法相比展现出良好的有效性。代码发布于https://github.com/ielab/dense-screening-feedback。