Model degrees of freedom ($\df$) is a fundamental concept in statistics because it quantifies the flexibility of a fitting procedure and is indispensable in model selection. To investigate the gap between $\df$ and the number of independent variables in the fitting procedure, \textcite{tibshiraniDegreesFreedomModel2015} introduced the \emph{search degrees of freedom} ($\sdf$) concept to account for the search cost during model selection. However, this definition has two limitations: it does not consider fitting procedures in augmented spaces and does not use the same fitting procedure for $\sdf$ and $\df$. We propose a \emph{modified search degrees of freedom} ($\msdf$) to directly account for the cost of searching in either original or augmented spaces. We check this definition for various fitting procedures, including classical linear regressions, spline methods, adaptive regressions (the best subset and the lasso), regression trees, and multivariate adaptive regression splines (MARS). In many scenarios when $\sdf$ is applicable, $\msdf$ reduces to $\sdf$. However, for certain procedures like the lasso, $\msdf$ offers a fresh perspective on search costs. For some complex procedures like MARS, the $\df$ has been pre-determined during model fitting, but the $\df$ of the final fitted procedure might differ from the pre-determined one. To investigate this discrepancy, we introduce the concepts of \emph{nominal} $\df$ and \emph{actual} $\df$, and define the property of \emph{self-consistency}, which occurs when there is no gap between these two $\df$'s. We propose a correction procedure for MARS to align these two $\df$'s, demonstrating improved fitting performance through extensive simulations and two real data applications.
翻译:模型自由度($\df$)是统计学中的基本概念,因为它量化了拟合过程的灵活性,并且在模型选择中不可或缺。为探究$\df$与拟合过程中自变量数量之间的差异,\textcite{tibshiraniDegreesFreedomModel2015}引入了\textbf{搜索自由度}($\sdf$)概念以解释模型选择过程中的搜索成本。然而,该定义存在两个局限性:未考虑在增广空间中的拟合过程,且未对$\sdf$与$\df$使用相同的拟合程序。我们提出\textbf{修正搜索自由度}($\msdf$)来直接解释在原始空间或增广空间中的搜索成本。我们针对多种拟合程序验证了该定义,包括经典线性回归、样条方法、自适应回归(最优子集与lasso)、回归树以及多元自适应回归样条(MARS)。在$\sdf$适用的许多场景中,$\msdf$可简化为$\sdf$。然而,对于如lasso等特定方法,$\msdf$为搜索成本提供了新的视角。对于MARS等复杂方法,其$\df$在模型拟合过程中已预先确定,但最终拟合过程的$\df$可能与预先确定的自由度存在差异。为探究此差异,我们引入了\textbf{名义}$\df$与\textbf{实际}$\df$的概念,并定义了\textbf{自一致性}属性——当两种自由度之间无差异时即满足该属性。我们提出了一种针对MARS的校正程序以使两种自由度对齐,并通过大量模拟实验与两个实际数据应用证明了其改进的拟合性能。