The stochastic generalised linear bandit is a well-understood model for sequential decision-making problems, with many algorithms achieving near-optimal regret guarantees under immediate feedback. However, the stringent requirement for immediate rewards is unmet in many real-world applications where the reward is almost always delayed. We study the phenomenon of delayed rewards in generalised linear bandits in a theoretical manner. We show that a natural adaptation of an optimistic algorithm to the delayed feedback achieves a regret bound where the penalty for the delays is independent of the horizon. This result significantly improves upon existing work, where the best known regret bound has the delay penalty increasing with the horizon. We verify our theoretical results through experiments on simulated data.

翻译：随机广义线性老虎机是处理序贯决策问题的经典模型，在即时反馈条件下已有多种算法实现了近乎最优的遗憾界。然而，许多实际应用中奖励几乎总是存在延迟，使得对即时奖励的严苛要求无法满足。本文从理论角度研究广义线性老虎机中的奖励延迟现象。研究表明，将乐观算法自然适配至延迟反馈场景后，所获遗憾界中由延迟导致的惩罚项与时间范围无关。该结果显著优于现有工作——当前已知最优遗憾界中，延迟惩罚会随时间范围增加而增大。我们通过模拟数据实验验证了理论结果。