In this paper, we consider a general observation model for restless multi-armed bandit problems. The operation of the player is based on the past observation history that is limited (partial) and error-prone due to resource constraints or environmental or intrinsic noises. By establishing a general probabilistic model for dynamics of the observation process, we formulate the problem as a restless bandit with an infinite high-dimensional belief state space. We apply the achievable region method with partial conservation law (PCL) to the infinite-state problem and analyze its indexability and priority index (Whittle index). Finally, we propose an approximation process to transform the problem into which the AG algorithm of Niño-Mora (2001) for finite-state problems can be applied. Numerical experiments show that our algorithm has excellent performance.

翻译：本文针对不安定多臂老虎机问题提出了一种通用的观测模型。由于资源限制或环境及内在噪声的影响，决策者的操作基于有限（部分）且易出错的过往观测历史。通过建立观测过程动态的一般概率模型，我们将该问题表述为具有无限高维信念状态空间的不安定老虎机问题。我们将可实现区域方法与部分守恒定律（PCL）应用于这一无限状态问题，并分析其可索引性及优先级指数（Whittle指数）。最后，我们提出一种近似转换方法，将问题转化为可应用Niño-Mora（2001）针对有限状态问题提出的AG算法的形式。数值实验表明，我们所提算法具有优异的性能。