In this paper, we study kernelized bandits with distributed biased feedback. This problem is motivated by several real-world applications (such as dynamic pricing, cellular network configuration, and policy making), where users from a large population contribute to the reward of the action chosen by a central entity, but it is difficult to collect feedback from all users. Instead, only biased feedback (due to user heterogeneity) from a subset of users may be available. In addition to such partial biased feedback, we are also faced with two practical challenges due to communication cost and computation complexity. To tackle these challenges, we carefully design a new \emph{distributed phase-then-batch-based elimination (\texttt{DPBE})} algorithm, which samples users in phases for collecting feedback to reduce the bias and employs \emph{maximum variance reduction} to select actions in batches within each phase. By properly choosing the phase length, the batch size, and the confidence width used for eliminating suboptimal actions, we show that \texttt{DPBE} achieves a sublinear regret of $\tilde{O}(T^{1-\alpha/2}+\sqrt{\gamma_T T})$, where $\alpha\in (0,1)$ is the user-sampling parameter one can tune. Moreover, \texttt{DPBE} can significantly reduce both communication cost and computation complexity in distributed kernelized bandits, compared to some variants of the state-of-the-art algorithms (originally developed for standard kernelized bandits). Furthermore, by incorporating various \emph{differential privacy} models (including the central, local, and shuffle models), we generalize \texttt{DPBE} to provide privacy guarantees for users participating in the distributed learning process. Finally, we conduct extensive simulations to validate our theoretical results and evaluate the empirical performance.

翻译：本文研究分布式有偏反馈下的核化赌博机问题。该问题源于多个实际应用场景（如动态定价、蜂窝网络配置和决策制定），其中大量用户共同贡献于中央实体所选动作的奖励，但难以收集所有用户的反馈，仅能获取部分用户存在偏差的反馈（由于用户异质性）。除这种部分有偏反馈外，通信成本和计算复杂度也构成两大实际挑战。为解决这些问题，我们精心设计了一种新型的分布式分阶段批处理消除（DPBE）算法。该算法分阶段采样用户以收集反馈从而减少偏差，并在每个阶段内采用最大方差缩减方法批量选择动作。通过合理选择阶段长度、批量大小以及用于消除次优动作的置信区间宽度，我们证明DPBE可实现次线性遗憾界$\tilde{O}(T^{1-\alpha/2}+\sqrt{\gamma_T T})$，其中$\alpha\in(0,1)$为可调的用户采样参数。与现有最优算法（最初针对标准核化赌博机设计）的变体相比，DPBE能显著降低分布式核化赌博机的通信成本和计算复杂度。此外，通过集成多种差分隐私模型（包括中心化、本地化和混洗模型），我们将DPBE泛化以保障分布式学习过程中参与用户的隐私。最后，我们通过大量仿真实验验证理论结果并评估实际性能。