Decision-making problems of sequential nature, where decisions made in the past may have an impact on the future, are used to model many practically important applications. In some real-world applications, feedback about a decision is delayed and may arrive via partial rewards that are observed with different delays. Motivated by such scenarios, we propose a novel problem formulation called multi-armed bandits with generalized temporally-partitioned rewards. To formalize how feedback about a decision is partitioned across several time steps, we introduce $\beta$-spread property. We derive a lower bound on the performance of any uniformly efficient algorithm for the considered problem. Moreover, we provide an algorithm called TP-UCB-FR-G and prove an upper bound on its performance measure. In some scenarios, our upper bound improves upon the state of the art. We provide experimental results validating the proposed algorithm and our theoretical results.

翻译：具有序列性质的决策问题，其中过去的决策可能对未来产生影响，被用于建模许多实际重要应用。在某些现实场景中，关于决策的反馈存在延迟，并且可能通过具有不同延迟观测到的部分奖励到达。受此类场景启发，我们提出了一种名为“具有广义时间分段奖励的多臂赌博机”的新问题公式。为了形式化决策反馈如何在多个时间步上分段，我们引入了$\beta$-扩散性质。我们针对所考虑问题中任何均匀高效算法的性能推导了一个下界。此外，我们提出了一种名为TP-UCB-FR-G的算法，并证明了其性能度量的上界。在某些场景中，我们的上界优于现有技术。我们提供了验证所提出算法及其理论结果的实验结果。