We present RLStop, a novel Technology Assisted Review (TAR) stopping rule based on reinforcement learning that helps minimise the number of documents that need to be manually reviewed within TAR applications. RLStop is trained on example rankings using a reward function to identify the optimal point to stop examining documents. Experiments at a range of target recall levels on multiple benchmark datasets (CLEF e-Health, TREC Total Recall, and Reuters RCV1) demonstrated that RLStop substantially reduces the workload required to screen a document collection for relevance. RLStop outperforms a wide range of alternative approaches, achieving performance close to the maximum possible for the task under some circumstances.

翻译：我们提出了RLStop，一种基于强化学习的新型技术辅助审查（TAR）停止规则，旨在最小化TAR应用需要人工审查的文档数量。RLStop通过示例排序进行训练，利用奖励函数确定停止审查文档的最优时机。在多个基准数据集（CLEF e-Health、TREC Total Recall和Reuters RCV1）上，针对不同目标召回率水平的实验表明，RLStop能够显著降低筛选文档集合相关性所需的工作量。RLStop优于多种现有方法，在某些情况下其性能接近该任务的理论最优值。