Ensemble models in E-commerce combine predictions from multiple sub-models for ranking and revenue improvement. Industrial ensemble models are typically deep neural networks, following the supervised learning paradigm to infer conversion rate given inputs from sub-models. However, this process has the following two problems. Firstly, the point-wise scoring approach disregards the relationships between items and leads to homogeneous displayed results, while diversified display benefits user experience and revenue. Secondly, the learning paradigm focuses on the ranking metrics and does not directly optimize the revenue. In our work, we propose a new Learning-To-Ensemble (LTE) framework RAEGO, which replaces the ensemble model with a contextual Rank Aggregator (RA) and explores the best weights of sub-models by the Evaluator-Generator Optimization (EGO). To achieve the best online performance, we propose a new rank aggregation algorithm TournamentGreedy as a refinement of classic rank aggregators, which also produces the best average weighted Kendall Tau Distance (KTD) amongst all the considered algorithms with quadratic time complexity. Under the assumption that the best output list should be Pareto Optimal on the KTD metric for sub-models, we show that our RA algorithm has higher efficiency and coverage in exploring the optimal weights. Combined with the idea of Bayesian Optimization and gradient descent, we solve the online contextual Black-Box Optimization task that finds the optimal weights for sub-models given a chosen RA model. RA-EGO has been deployed in our online system and has improved the revenue significantly.

翻译：电子商务中的集成模型通过融合多个子模型的预测结果，实现排序优化与收入提升。工业级集成模型通常采用深度神经网络，遵循监督学习范式，根据子模型输入推断转化率。然而，该过程存在两个问题：首先，逐点评分方法忽视项目间关联，导致展示结果同质化，而多样性展示更有利于用户体验和收入提升；其次，该学习范式侧重排序指标，未直接优化收入。本研究提出了新型学习集成框架RA-EGO，该框架用上下文排序聚合器替代集成模型，并通过评估器-生成器优化方法探索子模型最优权重。为取得最佳在线性能，我们提出新型排序聚合算法TournamentGreedy作为经典聚合器的改进方案，该算法在二次时间复杂度下，在所有候选算法中取得了最优平均加权肯德尔秩距离。基于最优输出列表应在子模型KTD指标上达到帕累托最优的假设，我们证明了所提出的排序聚合算法在探索最优权重时具有更高的效率与覆盖率。结合贝叶斯优化与梯度下降思想，我们解决了给定排序聚合模型下为子模型寻找最优权重的在线上下文黑箱优化任务。RA-EGO已部署于在线系统，显著提升了收入。