Contextual multi-armed bandit is a fundamental learning framework for making a sequence of decisions, e.g., advertising recommendations for a sequence of arriving users. Recent works have shown that clustering these users based on the similarity of their learned preferences can accelerate the learning. However, prior work has primarily focused on the online setting, which requires continually collecting user data, ignoring the offline data widely available in many applications. To tackle these limitations, we study the offline clustering of bandits (Off-ClusBand) problem, which studies how to use the offline dataset to learn cluster properties and improve decision-making. The key challenge in Off-ClusBand arises from data insufficiency for users: unlike the online case where we continually learn from online data, in the offline case, we have a fixed, limited dataset to work from and thus must determine whether we have enough data to confidently cluster users together. To address this challenge, we propose two algorithms: Off-C2LUB, which we show analytically and experimentally outperforms existing methods under limited offline user data, and Off-CLUB, which may incur bias when data is sparse but performs well and nearly matches the lower bound when data is sufficient. We experimentally validate these results on both real and synthetic datasets.

翻译：上下文多臂赌博机是一种用于进行序列决策的基本学习框架，例如为一系列到达用户提供广告推荐。近期研究表明，基于学习到的用户偏好相似性对这些用户进行聚类可以加速学习过程。然而，先前工作主要关注在线场景，这需要持续收集用户数据，而忽略了在许多应用中广泛可用的离线数据。为解决这些局限性，我们研究了赌博机离线聚类问题，该问题探讨如何利用离线数据集学习聚类特性并改进决策。离线赌博机聚类的核心挑战源于用户数据的不足：与在线场景中持续从在线数据学习不同，在离线场景中我们只能基于固定且有限的数据集进行工作，因此必须判断是否有足够数据来可靠地将用户聚类在一起。为应对这一挑战，我们提出了两种算法：Off-C2LUB——我们通过分析和实验证明其在有限离线用户数据下优于现有方法；以及Off-CLUB——该算法在数据稀疏时可能产生偏差，但在数据充足时表现良好且几乎达到理论下界。我们在真实数据集和合成数据集上对这些结果进行了实验验证。