We consider the contextual bandit problem where at each time, the agent only has access to a noisy version of the context and the error variance (or an estimator of this variance). This setting is motivated by a wide range of applications where the true context for decision-making is unobserved, and only a prediction of the context by a potentially complex machine learning algorithm is available. When the context error is non-vanishing, classical bandit algorithms fail to achieve sublinear regret. We propose the first online algorithm in this setting with sublinear regret guarantees under mild conditions. The key idea is to extend the measurement error model in classical statistics to the online decision-making setting, which is nontrivial due to the policy being dependent on the noisy context observations. We further demonstrate the benefits of the proposed approach in simulation environments based on synthetic and real digital intervention datasets.

翻译：我们考虑上下文老虎机问题，其中在每个时间步，智能体只能访问到带噪声的上下文版本以及误差方差（或该方差的估计量）。该设定源于广泛的应用场景：真实决策上下文不可观测，仅能通过潜在复杂机器学习算法获得上下文预测值。当上下文误差非消失时，经典老虎机算法无法实现次线性遗憾。我们提出了该设定下首个具有温和条件次线性遗憾保证的在线算法。核心思路是将经典统计学中的测量误差模型扩展到在线决策设定中，由于策略依赖于含噪上下文观测，这一扩展具有非平凡性。我们进一步在基于合成数据与真实数字干预数据集的仿真环境中展示了所提方法的优势。