Contrastive learning is a core component of modern retrieval systems, but its effectiveness heavily relies on the quality of negative examples used during training. In this work, we present a systematic approach to improving dense retrieval for IKEA product search through structured negative sampling strategies and scalable LLM-as-a-judge relevance evaluation. Building on IKEA Search Engine's late-interaction retrieval architectures, we introduce two key contributions: (1) structured negative sampling strategies that leverage product hierarchical taxonomy and product attributes to generate semantically challenging negatives, and (2) a comprehensive LLM-based evaluation methodology for generating training data. Rather than relying on sparse human annotations or random sampling, our LLM-based evaluation system allocates a score for all candidate products against each query. Our methodology achieves +2.6\% average category accuracy on offline real user query experiments on the Canada market. However, our A/B test on long-tail queries showed no statistically significant differences in user engagement metrics between the improved and baseline models ($p > 0.05$). We trace this gap to user search behavior: 67\% of popular searches exhibit zero-click rates above 50\%, indicating that a substantial proportion of search sessions result in no product engagement regardless of result ranking. These findings underscore the importance of hard negative mining but also the need for grounding training data and offline evals in real user search behavior -- including query intent distribution and zero-click patterns -- to bridge the gap between offline retrieval quality and online user engagement.

翻译：对比学习是现代检索系统的核心组成部分，但其有效性高度依赖于训练过程中使用的负例质量。本文提出了一种系统化方法，通过结构化负采样策略和可扩展的“以大语言模型为判官”相关性评估，改进宜家产品搜索中的稠密检索效果。基于宜家搜索引擎的后期交互检索架构，我们引入两项关键贡献：（1）利用产品层级分类体系和产品属性生成语义上更具挑战性的负例的结构化负采样策略；（2）一种用于生成训练数据的综合性大语言模型评估方法。该方法无需依赖稀疏的人工标注或随机采样，而是为所有候选产品针对每个查询分配评分。在加拿大市场的离线真实用户查询实验中，我们的方法在平均类别准确率上获得了+2.6%的提升。然而，针对长尾查询的A/B测试显示，改进模型与基线模型在用户参与度指标上未出现统计学显著差异（p > 0.05）。我们将这一差距归因于用户搜索行为：67%的热门搜索的零点击率超过50%，表明大量搜索会话最终未产生任何产品交互，无论结果排序如何。这些发现不仅强调了难负例挖掘的重要性，更凸显了将训练数据与离线评估建立在真实用户搜索行为（包括查询意图分布与零点击模式）基础上的必要性，以弥合离线检索质量与在线用户参与度之间的鸿沟。