Representation learning plays a crucial role in automated feature selection, particularly in the context of high-dimensional data, where non-parametric methods often struggle. In this study, we focus on supervised learning scenarios where the pertinent information resides within a lower-dimensional linear subspace of the data, namely the multi-index model. If this subspace were known, it would greatly enhance prediction, computation, and interpretation. To address this challenge, we propose a novel method for linear feature learning with non-parametric prediction, which simultaneously estimates the prediction function and the linear subspace. Our approach employs empirical risk minimisation, augmented with a penalty on function derivatives, ensuring versatility. Leveraging the orthogonality and rotation invariance properties of Hermite polynomials, we introduce our estimator, named RegFeaL. By utilising alternative minimisation, we iteratively rotate the data to improve alignment with leading directions and accurately estimate the relevant dimension in practical settings. We establish that our method yields a consistent estimator of the prediction function with explicit rates. Additionally, we provide empirical results demonstrating the performance of RegFeaL in various experiments.

翻译：表示学习在自动特征选择中扮演着关键角色，尤其是在高维数据场景下，非参数方法常面临挑战。本研究聚焦于监督学习场景，其中关键信息存在于数据的低维线性子空间中，即多指标模型。若该子空间已知，将大幅提升预测、计算和解释能力。为解决这一问题，我们提出一种结合非参数预测的线性特征学习方法，可同时估计预测函数和线性子空间。该方法采用经验风险最小化，并辅以函数导数的惩罚项，确保其通用性。利用埃尔米特多项式的正交性和旋转不变性，我们提出名为RegFeaL的估计量。通过交替最小化策略，我们迭代旋转数据以增强与主导方向的对齐，并在实际场景中准确估计相关维度。理论证明该方法能以显式收敛率得到预测函数的一致估计量。此外，我们通过多种实验提供了验证RegFeaL性能的实证结果。