The asymptotic mean squared test error and sensitivity of the Random Features Regression model (RFR) have been recently studied. We build on this work and identify in closed-form the family of Activation Functions (AFs) that minimize a combination of the test error and sensitivity of the RFR under different notions of functional parsimony. We find scenarios under which the optimal AFs are linear, saturated linear functions, or expressible in terms of Hermite polynomials. Finally, we show how using optimal AFs impacts well-established properties of the RFR model, such as its double descent curve, and the dependency of its optimal regularization parameter on the observation noise level.

翻译：近期，学者们对随机特征回归模型（RFR）的渐近均方测试误差与敏感性进行了研究。本文基于这些研究，以闭式形式识别出在功能简约的不同约束下，可使RFR测试误差与敏感性联合最小化的激活函数（AFs）族。我们发现了最优激活函数呈现线性函数、饱和线性函数或由埃尔米特多项式可表形式的具体场景。最后，我们揭示了最优激活函数对RFR模型固有性质（如双下降曲线及最优正则化参数对观测噪声水平的依赖性）的影响机制。