Recommendation strategies are typically evaluated by using previously logged data, employing off-policy evaluation methods to estimate their expected performance. However, for strategies that present users with slates of multiple items, the resulting combinatorial action space renders many of these methods impractical. Prior work has developed estimators that leverage the structure in slates to estimate the expected off-policy performance, but the estimation of the entire performance distribution remains elusive. Estimating the complete distribution allows for a more comprehensive evaluation of recommendation strategies, particularly along the axes of risk and fairness that employ metrics computable from the distribution. In this paper, we propose an estimator for the complete off-policy performance distribution for slates and establish conditions under which the estimator is unbiased and consistent. This builds upon prior work on off-policy evaluation for slates and off-policy distribution estimation in reinforcement learning. We validate the efficacy of our method empirically on synthetic data as well as on a slate recommendation simulator constructed from real-world data (MovieLens-20M). Our results show a significant reduction in estimation variance and improved sample efficiency over prior work across a range of slate structures.

翻译：推荐策略通常利用历史记录数据，通过离策略评估方法估计其预期性能。然而，对于向用户呈现多项目列表的策略，其组合动作空间使得许多方法难以应用。已有工作开发了利用列表结构估计离策略预期性能的估计器，但完整性能分布的估计仍难以实现。估计完整分布可对推荐策略进行更全面评估，特别是在风险与公平性维度上，这些维度依赖于可从分布中计算的指标。本文提出了一种面向列表的完整离策略性能分布估计器，并建立了该估计器无偏且一致的适用条件。该工作基于列表离策略评估与强化学习离线分布估计的前期研究。我们在合成数据及基于真实数据（MovieLens-20M）构建的列表推荐模拟器上验证了方法的有效性。结果表明，在多种列表结构下，相较于现有方法，我们显著降低了估计方差并提升了样本效率。