Representation learning plays a crucial role in automated feature selection, particularly in the context of high-dimensional data, where non-parametric methods often struggle. In this study, we focus on supervised learning scenarios where the pertinent information resides within a lower-dimensional linear subspace of the data, namely the multi-index model. If this subspace were known, it would greatly enhance prediction, computation, and interpretation. To address this challenge, we propose a novel method for linear feature learning with non-parametric prediction, which simultaneously estimates the prediction function and the linear subspace. Our approach employs empirical risk minimisation, augmented with a penalty on function derivatives, ensuring versatility. Leveraging the orthogonality and rotation invariance properties of Hermite polynomials, we introduce our estimator, named RegFeaL. By utilising alternative minimisation, we iteratively rotate the data to improve alignment with leading directions and accurately estimate the relevant dimension in practical settings. We establish that our method yields a consistent estimator of the prediction function with explicit rates. Additionally, we provide empirical results demonstrating the performance of RegFeaL in various experiments.

翻译：表示学习在自动特征选择中起着关键作用，尤其是在高维数据背景下，非参数方法常受限于维度灾难。本研究聚焦于监督学习场景，其中相关信息位于数据的低维线性子空间中，即多指标模型。若该子空间已知，将显著提升预测、计算和解释能力。为此，我们提出一种融合非参数预测的线性特征学习方法，可同时估计预测函数与线性子空间。该方法采用经验风险最小化，并通过函数导数惩罚项增强其通用性。利用埃尔米特多项式的正交性与旋转不变性，我们提出了名为RegFeaL的估计器。通过交替最小化策略，我们迭代旋转数据以提升与主方向的对齐度，并在实际场景中精确估计相关维度。理论分析表明，该方法能以显式收敛速度实现预测函数的一致估计。此外，多组实验验证了RegFeaL的性能表现。