Recommendation strategies are typically evaluated by using previously logged data, employing off-policy evaluation methods to estimate their expected performance. However, for strategies that present users with slates of multiple items, the resulting combinatorial action space renders many of these methods impractical. Prior work has developed estimators that leverage the structure in slates to estimate the expected off-policy performance, but the estimation of the entire performance distribution remains elusive. Estimating the complete distribution allows for a more comprehensive evaluation of recommendation strategies, particularly along the axes of risk and fairness that employ metrics computable from the distribution. In this paper, we propose an estimator for the complete off-policy performance distribution for slates and establish conditions under which the estimator is unbiased and consistent. This builds upon prior work on off-policy evaluation for slates and off-policy distribution estimation in reinforcement learning. We validate the efficacy of our method empirically on synthetic data as well as on a slate recommendation simulator constructed from real-world data (MovieLens-20M). Our results show a significant reduction in estimation variance and improved sample efficiency over prior work across a range of slate structures.

翻译：推荐策略通常通过使用历史记录数据进行评估，并借助离线评估方法来估计其预期性能。然而，对于向用户展示包含多个项目的数量（slates）的策略而言，由此产生的组合动作空间使得许多方法变得不切实际。先前的工作已开发出利用数量结构来估计预期离线性能的估计器，但对完整性能分布的估计仍然难以实现。估计完整分布有助于对推荐策略进行更全面的评估，尤其是在风险与公平性维度上，这些维度依赖于可从分布中计算的指标。本文提出了一种针对数量的完整离线性能分布估计器，并确立了该估计器无偏且一致的成立条件。这一工作建立在先前关于数量离线评估和强化学习中离线分布估计的研究基础之上。我们在合成数据以及基于真实数据（MovieLens-20M）构建的数量推荐模拟器上实证验证了方法的有效性。结果表明，相较于先前工作，我们的方法在多种数量结构下显著降低了估计方差并提升了样本效率。