We propose a generic variance-reduced algorithm, which we call MUltiple RANdomized Algorithm (MURANA), for minimizing a sum of several smooth functions plus a regularizer, in a sequential or distributed manner. Our method is formulated with general stochastic operators, which allow us to model various strategies for reducing the computational complexity. For example, MURANA supports sparse activation of the gradients, and also reduction of the communication load via compression of the update vectors. This versatility allows MURANA to cover many existing randomization mechanisms within a unified framework, which also makes it possible to design new methods as special cases.

翻译：我们提出了一种通用方差缩减算法，称为多重随机化算法（MURANA），用于以串行或分布式方式最小化多个光滑函数与正则化项的和。该方法基于通用随机算子构建，可建模多种降低计算复杂度的策略。例如，MURANA支持梯度稀疏激活，并通过压缩更新向量降低通信负载。这种灵活性使得MURANA能够将多种现有随机化机制纳入统一框架，同时也可作为特例设计新方法。