A general, {\em rectangular} kernel matrix may be defined as $K_{ij} = \kappa(x_i,y_j)$ where $\kappa(x,y)$ is a kernel function and where $X=\{x_i\}_{i=1}^m$ and $Y=\{y_i\}_{i=1}^n$ are two sets of points. In this paper, we seek a low-rank approximation to a kernel matrix where the sets of points $X$ and $Y$ are large and are arbitrarily distributed, such as away from each other, ``intermingled'', identical, etc. Such rectangular kernel matrices may arise, for example, in Gaussian process regression where $X$ corresponds to the training data and $Y$ corresponds to the test data. In this case, the points are often high-dimensional. Since the point sets are large, we must exploit the fact that the matrix arises from a kernel function, and avoid forming the matrix, and thus ruling out most algebraic techniques. In particular, we seek methods that can scale linearly or nearly linear with respect to the size of data for a fixed approximation rank. The main idea in this paper is to {\em geometrically} select appropriate subsets of points to construct a low rank approximation. An analysis in this paper guides how this selection should be performed.
翻译:定义一般矩形核矩阵为 $K_{ij} = \kappa(x_i,y_j)$,其中 $\kappa(x,y)$ 是核函数,$X=\{x_i\}_{i=1}^m$ 和 $Y=\{y_i\}_{i=1}^n$ 为两个点集。本文针对大规模且任意分布(如相互远离、交错混合、完全重合等)的点集 $X$ 和 $Y$,研究核矩阵的低秩逼近问题。此类矩形核矩阵常见于高斯过程回归场景($X$ 对应训练数据,$Y$ 对应测试数据),此时点集通常具有高维特征。由于点集规模庞大,必须利用矩阵源自核函数的特性,避免显式构建矩阵,从而排除大多数代数方法。具体而言,我们寻求在固定逼近秩下,计算复杂度与数据规模呈线性或近线性关系的方法。本文核心思想是通过**几何方法**选取合适的点子集构建低秩逼近,并通过理论分析指导该选择过程。