Technology Assisted Review stopping methods aim to ensure that no more documents are screened than necessary. Most existing approaches focus on achieving a target recall, which does not consider whether an information need has been met. This paper introduces two heuristic stopping methods that instead monitor whether screened documents contain enough information to make a decision. Evaluation on a standard dataset of Diagnostic Test Accuracy Systematic Reviews demonstrates that the proposed approaches substantially reduce the number of documents that need to be examined while, in the majority of cases, maintaining conclusions that are consistent with all evidence available.
翻译:技术辅助评审停止方法旨在确保筛选文献量不超过必要范围。现有方法大多聚焦于实现目标召回率,但未考虑信息需求是否已得到满足。本文提出两种基于启发式的停止方法,通过监控已筛选文献是否包含足够信息以做出决策。在诊断测试准确性系统评价标准数据集上的评估表明,所提出的方法在大多数情况下能够显著减少需审查的文献数量,同时保持与现有全部证据一致的结论。