Making image retrieval methods practical for real-world search applications requires significant progress in dataset scales, entity comprehension, and multimodal information fusion. In this work, we introduce \textbf{E}ntity-\textbf{D}riven \textbf{I}mage \textbf{S}earch (EDIS), a challenging dataset for cross-modal image search in the news domain. EDIS consists of 1 million web images from actual search engine results and curated datasets, with each image paired with a textual description. Unlike datasets that assume a small set of single-modality candidates, EDIS reflects real-world web image search scenarios by including a million multimodal image-text pairs as candidates. EDIS encourages the development of retrieval models that simultaneously address cross-modal information fusion and matching. To achieve accurate ranking results, a model must: 1) understand named entities and events from text queries, 2) ground entities onto images or text descriptions, and 3) effectively fuse textual and visual representations. Our experimental results show that EDIS challenges state-of-the-art methods with dense entities and a large-scale candidate set. The ablation study also proves that fusing textual features with visual features is critical in improving retrieval results.

翻译：使图像检索方法在实际搜索应用中具备实用性，需要在数据集规模、实体理解以及多模态信息融合方面取得显著进展。本文提出实体驱动图像检索（Entity-Driven Image Search, EDIS），这是一个面向新闻领域跨模态图像检索的挑战性数据集。EDIS包含100万张来自真实搜索引擎结果和精心整理数据集的网络图像，每张图像均配有文本描述。与假设候选集仅包含少量单模态数据的数据集不同，EDIS通过纳入百万级多模态图像-文本对作为候选数据，真实反映了网络图像搜索场景。该数据集旨在推动能同时处理跨模态信息融合与匹配的检索模型开发。为实现精准排序结果，模型需要：1）理解文本查询中的命名实体与事件，2）将实体映射至图像或文本描述，3）有效融合文本与视觉表征。实验结果表明，EDIS通过密集实体和大规模候选集对现有最优方法构成挑战。消融研究亦证明，将文本特征与视觉特征融合是提升检索效果的关键。