Embedding-based neural retrieval (EBR) is an effective search retrieval method in product search for tackling the vocabulary gap between customer search queries and products. The initial launch of our EBR system at Walmart yielded significant gains in relevance and add-to-cart rates [1]. However, despite EBR generally retrieving more relevant products for reranking, we have observed numerous instances of relevance degradation. Enhancing retrieval performance is crucial, as it directly influences product reranking and affects the customer shopping experience. Factors contributing to these degradations include false positives/negatives in the training data and the inability to handle query misspellings. To address these issues, we present several approaches to further strengthen the capabilities of our EBR model in terms of retrieval relevance. We introduce a Relevance Reward Model (RRM) based on human relevance feedback. We utilize RRM to remove noise from the training data and distill it into our EBR model through a multi-objective loss. In addition, we present the techniques to increase the performance of our EBR model, such as typo-aware training, and semi-positive generation. The effectiveness of our EBR is demonstrated through offline relevance evaluation, online AB tests, and successful deployments to live production. [1] Alessandro Magnani, Feng Liu, Suthee Chaidaroon, Sachin Yadav, Praveen Reddy Suram, Ajit Puthenputhussery, Sijie Chen, Min Xie, Anirudh Kashi, Tony Lee, et al. 2022. Semantic retrieval at walmart. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3495-3503.

翻译：基于嵌入的神经检索（EBR）是产品搜索中一种有效的检索方法，用于解决客户搜索查询与产品之间的词汇鸿沟问题。我们在沃尔玛首次推出的EBR系统在相关性和加购率方面取得了显著提升[1]。然而，尽管EBR通常能为重排序检索到更相关的产品，我们仍观察到许多相关性下降的实例。提升检索性能至关重要，因为它直接影响产品重排序并影响客户购物体验。导致这些下降的因素包括训练数据中的假阳性/假阴性，以及处理查询拼写错误的能力不足。为解决这些问题，我们提出了几种方法来进一步增强我们EBR模型在检索相关性方面的能力。我们引入了一个基于人工相关性反馈的相关性奖励模型（RRM）。我们利用RRM去除训练数据中的噪声，并通过多目标损失将其知识蒸馏到我们的EBR模型中。此外，我们提出了提高EBR模型性能的技术，例如拼写错误感知训练和半正样本生成。我们EBR的有效性通过离线相关性评估、在线AB测试以及成功部署到实际生产环境得到了验证。[1] Alessandro Magnani, Feng Liu, Suthee Chaidaroon, Sachin Yadav, Praveen Reddy Suram, Ajit Puthenputhussery, Sijie Chen, Min Xie, Anirudh Kashi, Tony Lee, et al. 2022. Semantic retrieval at walmart. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3495-3503.