In large-scale recommender systems, retrieving top N relevant candidates accurately with resource constrain is crucial. To evaluate the performance of such retrieval models, Recall@N, the frequency of positive samples being retrieved in the top N ranking, is widely used. However, most of the conventional loss functions for retrieval models such as softmax cross-entropy and pairwise comparison methods do not directly optimize Recall@N. Moreover, those conventional loss functions cannot be customized for the specific retrieval size N required by each application and thus may lead to sub-optimal performance. In this paper, we proposed the Customizable Recall@N Optimization Loss (CROLoss), a loss function that can directly optimize the Recall@N metrics and is customizable for different choices of N. This proposed CROLoss formulation defines a more generalized loss function space, covering most of the conventional loss functions as special cases. Furthermore, we develop the Lambda method, a gradient-based method that invites more flexibility and can further boost the system performance. We evaluate the proposed CROLoss on two public benchmark datasets. The results show that CROLoss achieves SOTA results over conventional loss functions for both datasets with various choices of retrieval size N. CROLoss has been deployed onto our online E-commerce advertising platform, where a fourteen-day online A/B test demonstrated that CROLoss contributes to a significant business revenue growth of 4.75%.

翻译：在大规模推荐系统中，如何在资源约束下准确检索出前N个相关候选项目至关重要。为评估此类检索模型的性能，Recall@N（正样本在排序前N位中被检索出的频率）被广泛采用。然而，大多数传统检索模型损失函数（如softmax交叉熵和成对比较方法）并未直接优化Recall@N。此外，这些传统损失函数无法根据具体应用所需的检索规模N进行定制，可能导致次优性能。本文提出可定制Recall@N优化损失函数（CROLoss），该损失函数可直接优化Recall@N指标，并能针对不同N值进行定制。所提出的CROLoss公式定义了一个更广义的损失函数空间，将大多数传统损失函数涵盖为特例。进一步，我们开发了基于梯度的Lambda方法，该方法提供了更高灵活性，并能进一步提升系统性能。我们在两个公共基准数据集上评估了CROLoss，结果表明：针对不同检索规模N的选择，CROLoss在两个数据集上均取得了超越传统损失函数的最先进性能。CROLoss已部署于我们的在线电商广告平台，为期14天的在线A/B测试表明，CROLoss实现了4.75%的显著商业营收增长。