We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point $x$, to the subspace that captures the local variability of the function $f$. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.

翻译：本文引入了连续索引学习这一设定，其中多变量函数在每个点处仅沿少数方向变化。为实现高效估计，学习算法需要在每个点$x$附近自适应地调整至捕获函数$f$局部变异性的子空间。我们将此任务形式化为沿含噪声流形的核自适应问题，并提出局部EGOP学习算法——一种递归算法，该算法利用期望梯度外积二次型同时作为目标分布的度量和逆协方差矩阵。我们证明局部EGOP学习能自适应目标函数的正则性，表明在监督噪声流形假设下，该算法对任意高维噪声均可达到本征维度的学习速率。实证研究中，我们将本算法与深度学习的特征学习能力进行对比，并在连续单索引设定中展示了相较于双层神经网络回归质量的提升。