Kernel herding belongs to a family of deterministic quadratures that seek to minimize the worst-case integration error over a reproducing kernel Hilbert space (RKHS). In spite of strong experimental support, it has revealed difficult to prove that this worst-case error decreases at a faster rate than the standard square root of the number of quadrature nodes, at least in the usual case where the RKHS is infinite-dimensional. In this theoretical paper, we study a joint probability distribution over quadrature nodes, whose support tends to minimize the same worst-case error as kernel herding. We prove that it does outperform i.i.d. Monte Carlo, in the sense of coming with a tighter concentration inequality on the worst-case integration error. While not improving the rate yet, this demonstrates that the mathematical tools of the study of Gibbs measures can help understand to what extent kernel herding and its variants improve on computationally cheaper methods. Moreover, we provide early experimental evidence that a faster rate of convergence, though not worst-case, is likely.
翻译:核放牧属于一类确定性求积方法,旨在最小化再生核希尔伯特空间(RKHS)上的最坏情况积分误差。尽管有强有力的实验支持,但很难证明这种最坏情况误差的下降速度快于求积节点数量的标准平方根,至少在RKHS为无限维的常见情况下如此。在这篇理论性论文中,我们研究了求积节点上的联合概率分布,其支撑集倾向于最小化与核放牧相同的最坏情况误差。我们证明,该方法确实优于独立同分布蒙特卡洛方法,即对最坏情况积分误差具有更紧的集中不等式。尽管尚未改善收敛速率,但这表明研究吉布斯测度的数学工具有助于理解核放牧及其变体在何种程度上优于计算成本更低的方法。此外,我们提供了初步实验证据,表明更快的收敛速率(尽管并非最坏情况)是可能存在的。