We propose Grab-UCB, a graph-kernel multi-arms bandit algorithm to learn online the optimal source placement in large scale networks, such that the reward obtained from a priori unknown network processes is maximized. The uncertainty calls for online learning, which suffers however from the curse of dimensionality. To achieve sample efficiency, we describe the network processes with an adaptive graph dictionary model, which typically leads to sparse spectral representations. This enables a data-efficient learning framework, whose learning rate scales with the dimension of the spectral representation model instead of the one of the network. We then propose Grab-UCB, an online sequential decision strategy that learns the parameters of the spectral representation while optimizing the action strategy. We derive the performance guarantees that depend on network parameters, which further influence the learning curve of the sequential decision strategy We introduce a computationally simplified solving method, Grab-arm-Light, an algorithm that walks along the edges of the polytope representing the objective function. Simulations results show that the proposed online learning algorithm outperforms baseline offline methods that typically separate the learning phase from the testing one. The results confirm the theoretical findings, and further highlight the gain of the proposed online learning strategy in terms of cumulative regret, sample efficiency and computational complexity.

翻译：我们提出Grab-UCB，一种基于图核的多臂赌博机算法，用于在线学习大规模网络中的最优源点布局，以最大化从先验未知的网络过程中获得的奖励。这种不确定性要求进行在线学习，然而在线学习受到维度灾难的困扰。为了实现样本高效性，我们采用自适应图字典模型描述网络过程，该模型通常能产生稀疏的谱表示。这构建了一个数据高效的学习框架，其学习速率取决于谱表示模型的维度而非网络的维度。随后，我们提出Grab-UCB这一在线序贯决策策略，该策略在优化动作策略的同时学习谱表示的参数。我们推导了依赖于网络参数的性能保证，这些参数进一步影响序贯决策策略的学习曲线。我们引入一种计算简化的求解方法Grab-arm-Light，该算法沿表示目标函数的多面体边进行行走。仿真结果表明，所提出的在线学习算法优于通常将学习阶段与测试阶段分离的基线离线方法。这些结果印证了理论发现，并进一步凸显了所提出的在线学习策略在累积遗憾、样本效率和计算复杂度方面的优势。