The \emph{Fast Gaussian Transform} (FGT) enables subquadratic-time multiplication of an $n\times n$ Gaussian kernel matrix $\mathsf{K}_{i,j}= \exp ( - \| x_i - x_j \|_2^2 ) $ with an arbitrary vector $h \in \mathbb{R}^n$, where $x_1,\dots, x_n \in \mathbb{R}^d$ are a set of \emph{fixed} source points. This kernel plays a central role in machine learning and random feature maps. Nevertheless, in most modern data analysis applications, datasets are dynamically changing (yet often have low rank), and recomputing the FGT from scratch in (kernel-based) algorithms incurs a major computational overhead ($\gtrsim n$ time for a single source update $\in \mathbb{R}^d$). These applications motivate a \emph{dynamic FGT} algorithm, which maintains a dynamic set of sources under \emph{kernel-density estimation} (KDE) queries in \emph{sublinear time} while retaining Mat-Vec multiplication accuracy and speed. Assuming the dynamic data-points $x_i$ lie in a (possibly changing) $k$-dimensional subspace ($k\leq d$), our main result is an efficient dynamic FGT algorithm, supporting the following operations in $\log^{O(k)}(n/\varepsilon)$ time: (1) Adding or deleting a source point, and (2) Estimating the ``kernel-density'' of a query point with respect to sources with $\varepsilon$ additive accuracy. The core of the algorithm is a dynamic data structure for maintaining the \emph{projected} ``interaction rank'' between source and target boxes, decoupled into finite truncation of Taylor and Hermite expansions.
翻译:快速高斯变换(FGT)能够以次二次时间计算$n\times n$高斯核矩阵$\mathsf{K}_{i,j}= \exp ( - \| x_i - x_j \|_2^2 ) $与任意向量$h \in \mathbb{R}^n$的乘法,其中$x_1,\dots, x_n \in \mathbb{R}^d$为一组固定源点。该核函数在机器学习与随机特征映射中具有核心地位。然而,在大多数现代数据分析应用中,数据集是动态变化的(通常具有低秩特性),在基于核的算法中从头重新计算FGT会带来重大计算开销(单次源点更新$\in \mathbb{R}^d$需$\gtrsim n$时间)。这些应用推动了一种动态FGT算法的发展,该算法在维持矩阵-向量乘法精度与速度的同时,能够以次线性时间维护动态源点集合并响应核密度估计(KDE)查询。假设动态数据点$x_i$位于一个(可能变化的)$k$维子空间($k\leq d$),我们的主要成果是一种高效的动态FGT算法,支持以下操作,且时间复杂度为$\log^{O(k)}(n/\varepsilon)$:(1)添加或删除一个源点;(2)以$\varepsilon$加法精度估计查询点相对于源点的“核密度”。该算法核心是一种动态数据结构,用于维护源点与目标盒之间“投影交互秩”,通过泰勒展开与埃尔米特展开的有限截断实现解耦。