An accurate assessment of a model's complexity is crucial for topics such as interpretation, generalization, and model selection. However, most existing complexity measures either rely on heuristic assumptions or are computationally prohibitive. In this paper, we present a mathematically rigorous yet easy-to-compute measure of model complexity that is based on the similarities between the model gradients across inputs. It is thus well-defined for any parametric model, but also for kernel-based non-parametric models. We prove that our measure of complexity generalizes model-specific complexity measures such as polynomial degree (for polynomial regression), kernel length scale (for Matérn kernels), number of neighbors (for k-nearest neighbors), number of splits (for decision trees), and number of trees (for random forests). We also use our measure to obtain new insights into the double descent phenomenon for random Fourier features, random forests, neural networks, and gradient boosting.

翻译：准确评估模型复杂度对于模型解释、泛化以及模型选择等课题至关重要。然而，大多数现有的复杂度度量方法要么依赖于启发式假设，要么计算上过于昂贵。在本文中，我们提出了一种数学上严谨且易于计算的模型复杂度度量方法，该方法基于模型在不同输入上的梯度之间的相似性。因此，它对任何参数化模型以及基于核的非参数化模型都有良好定义。我们证明了我们的复杂度度量方法能够推广模型特定的复杂度度量，例如多项式回归中的多项式次数、Matérn 核中的核长度尺度、k近邻中的邻居数、决策树中的分裂次数以及随机森林中的树数量。我们还利用我们的度量方法获得了关于随机傅里叶特征、随机森林、神经网络和梯度提升的双重下降现象的新见解。