We consider distributed online min-max resource allocation with a set of parallel agents and a parameter server. Our goal is to minimize the pointwise maximum over a set of time-varying and decreasing cost functions, without a priori information about these functions. We propose a novel online algorithm, termed Distributed Online resource Re-Allocation (DORA), where non-stragglers learn to relinquish resource and share resource with stragglers. A notable feature of DORA is that it does not require gradient calculation or projection operation, unlike most existing online optimization strategies. This allows it to substantially reduce the computation overhead in large-scale and distributed networks. We analyze the worst-case performance of DORA and derive an upper bound on its dynamic regret for non-convex functions. We further consider an application to the bandwidth allocation problem in distributed online machine learning. Our numerical study demonstrates the efficacy of the proposed solution and its performance advantage over gradient- and/or projection-based resource allocation algorithms in reducing wall-clock time.

翻译：我们考虑包含一组并行智能体与参数服务器的分布式在线最小-最大资源分配问题。目标是在无先验信息的情况下，最小化一组时变递减代价函数的逐点最大值。我们提出一种新颖的在线算法——分布式在线资源重分配（DORA），其中非落后智能体学习释放资源并与落后智能体共享资源。DORA的显著特征在于无需像大多数现有在线优化策略那样计算梯度或执行投影操作，从而显著降低大规模分布式网络中的计算开销。我们分析了DORA的最坏情况性能，并推导出非凸函数动态遗憾的上界。进一步考虑分布式在线机器学习中的带宽分配应用问题。数值研究证明了所提方法的有效性，以及在减少挂钟时间方面相对于基于梯度和/或投影的资源分配算法的性能优势。