Retrieving relevant plots from the book for a query is a critical task, which can improve the reading experience and efficiency of readers. Readers usually only give an abstract and vague description as the query based on their own understanding, summaries, or speculations of the plot, which requires the retrieval model to have a strong ability to estimate the abstract semantic associations between the query and candidate plots. However, existing information retrieval (IR) datasets cannot reflect this ability well. In this paper, we propose Plot Retrieval, a labeled dataset to train and evaluate the performance of IR models on the novel task Plot Retrieval. Text pairs in Plot Retrieval have less word overlap and more abstract semantic association, which can reflect the ability of the IR models to estimate the abstract semantic association, rather than just traditional lexical or semantic matching. Extensive experiments across various lexical retrieval, sparse retrieval, dense retrieval, and cross-encoder methods compared with human studies on Plot Retrieval show current IR models still struggle in capturing abstract semantic association between texts. Plot Retrieval can be the benchmark for further research on the semantic association modeling ability of IR models.

翻译：从书籍中检索与查询相关的情节是一项关键任务，能够提升读者的阅读体验与效率。读者通常基于自身理解、总结或对情节的推测，仅给出抽象模糊的描述作为查询，这就要求检索模型具备较强的能力，以评估查询与候选情节之间的抽象语义关联。然而，现有的信息检索数据集无法很好地反映这一能力。本文提出情节检索这一标注数据集，用于训练和评估信息检索模型在新任务“情节检索”上的性能。情节检索中的文本对词汇重叠度较低，且具有更强的抽象语义关联，这能反映信息检索模型评估抽象语义关联的能力，而非仅仅依赖传统的词汇或语义匹配。通过将各种词汇检索、稀疏检索、密集检索及交叉编码器方法，与基于情节检索的人为研究进行广泛对比实验，结果表明当前的检索模型在捕捉文本间的抽象语义关联方面仍存在困难。情节检索可作为进一步研究信息检索模型语义关联建模能力的基准数据集。