We consider the contextual bandit problem where at each time, the agent only has access to a noisy version of the context and the error variance (or an estimator of this variance). This setting is motivated by a wide range of applications where the true context for decision-making is unobserved, and only a prediction of the context by a potentially complex machine learning algorithm is available. When the context error is non-vanishing, classical bandit algorithms fail to achieve sublinear regret. We propose the first online algorithm in this setting with sublinear regret guarantees under mild conditions. The key idea is to extend the measurement error model in classical statistics to the online decision-making setting, which is nontrivial due to the policy being dependent on the noisy context observations. We further demonstrate the benefits of the proposed approach in simulation environments based on synthetic and real digital intervention datasets.

翻译：我们考虑上下文赌臂问题，其中在每个时间步，智能体仅能获取带噪声的上下文版本以及误差方差（或该方差的估计量）。该设定源于广泛的实际应用场景：决策所需的真实上下文不可观测，仅能通过潜在复杂的机器学习算法获得上下文的预测值。当上下文误差非消失时，经典赌臂算法无法实现次线性遗憾。我们首次提出在该设定下具有次线性遗憾保证的在线算法，且该算法仅需温和条件。核心思想是将经典统计学中的测量误差模型扩展到在线决策场景，其非平凡性源于策略对噪声上下文观测的依赖性。我们进一步在基于合成数据集和真实数字干预数据集的仿真环境中验证了所提方法的优势。