We introduce the cram method as a general statistical framework for evaluating the final learned policy from a multi-armed contextual bandit algorithm, using the dataset generated by the same bandit algorithm. The proposed on-policy evaluation methodology differs from most existing methods that focus on off-policy performance evaluation of contextual bandit algorithms. Cramming utilizes an entire bandit sequence through a single pass of data, leading to both statistically and computationally efficient evaluation. We prove that if a bandit algorithm satisfies a certain stability condition, the resulting crammed evaluation estimator is consistent and asymptotically normal under mild regularity conditions. Furthermore, we show that this stability condition holds for commonly used linear contextual bandit algorithms, including epsilon-greedy, Thompson Sampling, and Upper Confidence Bound algorithms. Using both synthetic and publicly available datasets, we compare the empirical performance of cramming with the state-of-the-art methods. The results demonstrate that the proposed cram method reduces the evaluation standard error by approximately 40% relative to off-policy evaluation methods while preserving unbiasedness and valid confidence interval coverage.

翻译：本文提出了一种称为压缩方法的通用统计框架，用于评估从多臂上下文赌博机算法中习得的最终策略，该评估使用由同一赌博机算法生成的数据集。所提出的基于策略的评估方法不同于大多数专注于上下文赌博机算法离策略性能评估的现有方法。压缩方法通过单次数据遍历利用整个赌博机序列，从而实现统计和计算上均高效的评估。我们证明，如果一个赌博机算法满足特定的稳定性条件，则在温和的正则性条件下，所得的压缩评估估计量具有一致性和渐近正态性。此外，我们证明了这种稳定性条件对于常用的线性上下文赌博机算法（包括epsilon-greedy、Thompson Sampling和Upper Confidence Bound算法）均成立。通过使用合成数据集和公开可用的数据集，我们将压缩方法与最先进的方法进行了实证性能比较。结果表明，所提出的压缩方法相对于离策略评估方法，在保持无偏性和有效置信区间覆盖的同时，将评估标准误差降低了约40%。