Screening prioritisation in medical systematic reviews aims to rank the set of documents retrieved by complex Boolean queries. Prioritising the most important documents ensures that subsequent review steps can be carried out more efficiently and effectively. The current state of the art uses the final title of the review as a query to rank the documents using BERT-based neural rankers. However, the final title is only formulated at the end of the review process, which makes this approach impractical as it relies on ex post facto information. At the time of screening, only a rough working title is available, with which the BERT-based ranker performs significantly worse than with the final title. In this paper, we explore alternative sources of queries for prioritising screening, such as the Boolean query used to retrieve the documents to be screened and queries generated by instruction-based generative large-scale language models such as ChatGPT and Alpaca. Our best approach is not only viable based on the information available at the time of screening, but also has similar effectiveness to the final title.
翻译:医学系统综述中的筛选优先排序旨在对由复杂布尔查询检索到的文档集进行排序。优先处理最重要的文档可确保后续综述步骤更高效、更有效地进行。当前最先进的方法使用综述的最终标题作为查询,借助基于BERT的神经排序器对文档进行排序。然而,最终标题仅在综述过程结束时才确定,这使得该方法依赖于事后信息,不切实际。在筛选时,仅有粗略的工作标题可用,而基于BERT的排序器使用该工作标题时的表现显著差于使用最终标题。在本文中,我们探索了用于筛选优先排序的替代查询来源,例如用于检索待筛选文档的布尔查询,以及由基于指令的生成式大规模语言模型(如ChatGPT和Alpaca)生成的查询。我们的最佳方法不仅基于筛选时可用的信息切实可行,而且其有效性也与使用最终标题相当。