Recent successes of massively overparameterized models have inspired a new line of work investigating the underlying conditions that enable overparameterized models to generalize well. This paper considers a framework where the possibly overparametrized model includes fake features, i.e., features that are present in the model but not in the data. We present a non-asymptotic high-probability bound on the generalization error of the ridge regression problem under the model misspecification of having fake features. Our highprobability results provide insights into the interplay between the implicit regularization provided by the fake features and the explicit regularization provided by the ridge parameter. Numerical results illustrate the trade-off between the number of fake features and how the optimal ridge parameter may heavily depend on the number of fake features.

翻译：近年来，过度参数化模型的成功催生了一系列研究，旨在探究这些模型能够实现良好泛化的潜在条件。本文考虑了一个框架，其中可能过度参数化的模型包含了虚假特征（即存在于模型中但不存在于数据中的特征）。我们提出了在存在虚假特征的模型误设定情况下，岭回归问题泛化误差的非渐近高概率界。这一高概率结果揭示了虚假特征提供的隐式正则化与岭参数提供的显式正则化之间的相互作用。数值结果展示了虚假特征数量与最优岭参数如何严重依赖于虚假特征数量之间的权衡关系。