The asymptotic mean squared test error and sensitivity of the Random Features Regression model (RFR) have been recently studied. We build on this work and identify in closed-form the family of Activation Functions (AFs) that minimize a combination of the test error and sensitivity of the RFR under different notions of functional parsimony. We find scenarios under which the optimal AFs are linear, saturated linear functions, or expressible in terms of Hermite polynomials. Finally, we show how using optimal AFs impacts well-established properties of the RFR model, such as its double descent curve, and the dependency of its optimal regularization parameter on the observation noise level.

翻译：最近对随机地貌回归模型(RFR)的无症状平均正方形测试错误和敏感度进行了研究,我们以这项工作为基础,以封闭式的形式确定了激活功能(AFs)的组合,将测试错误和RFR敏感度的结合最小化,并置于不同的功能狭义概念之下。我们发现了最佳的AF是线性、饱和线性功能,或以Hermite 多元度表示的情景。最后,我们展示了最佳AFs如何使用最佳AFs影响RFR模型的既定特性,如双向下移曲线,以及其最佳正规化参数对观测噪音水平的依赖性。