Contrastive learning has gained widespread adoption for retrieval tasks due to its minimal requirement for manual annotations. However, popular contrastive frameworks typically learn from binary relevance, making them ineffective at incorporating direct fine-grained rankings. In this paper, we curate a large-scale dataset featuring detailed relevance scores for each query-document pair to facilitate future research and evaluation. Subsequently, we propose Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking (GCL), which is designed to learn from fine-grained rankings beyond binary relevance scores. Our results show that GCL achieves a 94.5% increase in NDCG@10 for in-domain and 26.3 to 48.8% increases for cold-start evaluations, all relative to the CLIP baseline and involving ground truth rankings.

翻译：对比学习由于对人工标注需求极低，已在检索任务中获得广泛采用。然而，主流对比学习框架通常仅基于二元相关性进行学习，导致其难以融入直接的细粒度排序信息。本文构建了一个大规模数据集，其中包含每个查询-文档对的详细相关性评分，以推动未来研究与评估。在此基础上，我们提出了面向多模态检索与排序的广义对比学习（GCL），该方法旨在学习超越二元相关性评分的细粒度排序。实验结果表明，相较于CLIP基线且以真实排序为基准，GCL在领域内评估中实现了NDCG@10指标94.5%的提升，在冷启动评估中取得了26.3%至48.8%的提升。