In this paper, we study the problem of global reward maximization with only partial distributed feedback. This problem is motivated by several real-world applications (e.g., cellular network configuration, dynamic pricing, and policy selection) where an action taken by a central entity influences a large population that contributes to the global reward. However, collecting such reward feedback from the entire population not only incurs a prohibitively high cost but often leads to privacy concerns. To tackle this problem, we consider differentially private distributed linear bandits, where only a subset of users from the population are selected (called clients) to participate in the learning process and the central server learns the global model from such partial feedback by iteratively aggregating these clients' local feedback in a differentially private fashion. We then propose a unified algorithmic learning framework, called differentially private distributed phased elimination (DP-DPE), which can be naturally integrated with popular differential privacy (DP) models (including central DP, local DP, and shuffle DP). Furthermore, we prove that DP-DPE achieves both sublinear regret and sublinear communication cost. Interestingly, DP-DPE also achieves privacy protection ``for free'' in the sense that the additional cost due to privacy guarantees is a lower-order additive term. In addition, as a by-product of our techniques, the same results of ``free" privacy can also be achieved for the standard differentially private linear bandits. Finally, we conduct simulations to corroborate our theoretical results and demonstrate the effectiveness of DP-DPE.

翻译：本文研究仅有部分分布式反馈的全局奖励最大化问题。该问题源于多个实际应用场景（如蜂窝网络配置、动态定价和策略选择），其中中央实体采取的行动会影响贡献全局奖励的庞大群体。然而，从整个群体收集此类奖励反馈不仅成本高昂，还常引发隐私担忧。为解决此问题，我们考虑差分隐私分布式线性赌博机：仅从群体中选取部分用户（称为客户端）参与学习过程，中央服务器通过迭代聚合这些客户端的本地反馈（以差分隐私方式）从部分反馈中学习全局模型。我们提出统一算法学习框架——差分隐私分布式分阶段消除算法（DP-DPE），该框架可自然集成主流差分隐私模型（包括中心化DP、本地DP和洗牌DP）。进一步，我们证明DP-DPE既能实现亚线性遗憾，又能实现亚线性通信成本。值得注意的是，DP-DPE还实现“免费”隐私保护，即隐私保障带来的额外成本仅为低阶加性项。此外，作为技术副产品，“免费”隐私的相同结果也可应用于标准差分隐私线性赌博机。最后，通过仿真实验验证理论结果并展示DP-DPE的有效性。