Reducing the variance of the gradient estimator is known to improve the convergence rate of stochastic gradient-based optimization and sampling algorithms. One way of achieving variance reduction is to design importance sampling strategies. Recently, the problem of designing such schemes was formulated as an online learning problem with bandit feedback, and algorithms with sub-linear static regret were designed. In this work, we build on this framework and propose Avare, a simple and efficient algorithm for adaptive importance sampling for finite-sum optimization and sampling with decreasing step-sizes. Under standard technical conditions, we show that Avare achieves $\mathcal{O}(T^{2/3})$ and $\mathcal{O}(T^{5/6})$ dynamic regret for SGD and SGLD respectively when run with $\mathcal{O}(1/t)$ step sizes. We achieve this dynamic regret bound by leveraging our knowledge of the dynamics defined by the algorithm, and combining ideas from online learning and variance-reduced stochastic optimization. We validate empirically the performance of our algorithm and identify settings in which it leads to significant improvements.

翻译：降低梯度估计值差异的已知做法是提高基于梯度优化和抽样算法的趋同率,实现差异减少的方法之一是设计重要的抽样战略。最近,设计这种计划的问题被作为强盗反馈的在线学习问题提出来,并设计了具有亚线静态遗憾的算法。在这项工作中,我们以这个框架为基础,并提出了Avare,这是一个简单而高效的适应重要性抽样算法,用于有限和增量的优化和采样。在标准技术条件下,我们显示Avare实现了$\mathcal{O}(T ⁇ 2/3})$和$\mathcal{O}(T ⁇ 5/6}) 和$\mathcal{O}(T ⁇ 5/6}) 用于SGD和SGLD的动态遗憾,在使用$\mathcal{O}(1/t) 级尺码运行时,我们通过利用我们对算法界定的动态的知识,并结合在线学习和差异调整的优化思想,我们用经验验证了我们的算法的表现,并确定了它导致重大改进的环境。我们实现了这种强烈的遗憾。