We consider the contextual bandit problem where at each time, the agent only has access to a noisy version of the context and the error variance (or an estimator of this variance). This setting is motivated by a wide range of applications where the true context for decision-making is unobserved, and only a prediction of the context by a potentially complex machine learning algorithm is available. When the context error is non-diminishing, classical bandit algorithms fail to achieve sublinear regret. We propose the first online algorithm in this setting with sublinear regret compared to the appropriate benchmark. The key idea is to extend the measurement error model in classical statistics to the online decision-making setting, which is nontrivial due to the policy being dependent on the noisy context observations.

翻译：我们考虑情境强盗问题，其中在每个时间步，智能体只能访问上下文的含噪版本及误差方差（或该方差的估计量）。该设定源于广泛的实际应用场景——决策所需的真实上下文不可观测，仅能通过潜在复杂机器学习算法获得其预测值。当上下文误差非衰减时，经典强盗算法无法实现次线性遗憾。本文首次提出在该设定下具有与适当基准相比次线性遗憾的在线算法。核心思想在于将经典统计学中的测量误差模型扩展至在线决策场景，由于策略依赖含噪上下文观测，这一扩展具有非平凡性。