We study a decentralized multi-agent multi-armed bandit problem in which multiple clients are connected by time dependent random graphs provided by an environment. The reward distributions of each arm vary across clients and rewards are generated independently over time by an environment based on distributions that include both sub-exponential and sub-gaussian distributions. Each client pulls an arm and communicates with neighbors based on the graph provided by the environment. The goal is to minimize the overall regret of the entire system through collaborations. To this end, we introduce a novel algorithmic framework, which first provides robust simulation methods for generating random graphs using rapidly mixing Markov chains or the random graph model, and then combines an averaging-based consensus approach with a newly proposed weighting technique and the upper confidence bound to deliver a UCB-type solution. Our algorithms account for the randomness in the graphs, removing the conventional doubly stochasticity assumption, and only require the knowledge of the number of clients at initialization. We derive optimal instance-dependent regret upper bounds of order $\log{T}$ in both sub-gaussian and sub-exponential environments, and a nearly optimal mean-gap independent regret upper bound of order $\sqrt{T}\log T$ up to a $\log T$ factor. Importantly, our regret bounds hold with high probability and capture graph randomness, whereas prior works consider expected regret under assumptions and require more stringent reward distributions.

翻译：我们研究了一个去中心化多智能体多臂赌博机问题，其中多个客户端通过环境提供的时变随机图相互连接。每个臂的奖励分布在客户端之间各不相同，且奖励由环境基于包含次指数分布和次高斯分布的分布随时间独立生成。每个客户端根据环境提供的图拉取一个臂并与邻居通信。目标是通过协作最小化整个系统的总体遗憾。为此，我们提出了一种新颖的算法框架：首先利用快速混合马尔可夫链或随机图模型提供生成随机图的鲁棒模拟方法，然后将基于平均的共识方法与新提出的加权技术及上置信界相结合，输出一种UCB型解决方案。我们的算法考虑了图中的随机性，摒弃了传统的双随机性假设，且仅需在初始化时知道客户端数量。在次高斯和次指数环境下，我们推导出阶为$\log{T}$的最优实例相关遗憾上界，以及阶为$\sqrt{T}\log T$（至多$\log T$因子）的近乎最优均值间隙无关遗憾上界。重要的是，我们的遗憾界以高概率成立并捕捉了图随机性，而先前的工作仅在假设下考虑期望遗憾并要求更严格的奖励分布。