The effective degrees of freedom of penalized regression models quantify the actual amount of information used to generate predictions, playing a pivotal role in model evaluation and selection. Although a closed-form estimator is available for the Lasso penalty, adaptive extensions of widely used penalized approaches, including the Adaptive Lasso and Adaptive Group Lasso, have remained without analogous theoretical characterization. This paper presents the first unbiased estimator of the effective degrees of freedom for these methods, along with their main theoretical properties, for both orthogonal and non-orthogonal designs, derived within Stein's unbiased risk estimation framework. The resulting expressions feature inflation terms influenced by the regularization parameter, coefficient signs, and least-squares estimates. These advances enable more accurate model selection criteria and unbiased prediction error estimates, illustrated through synthetic and real data. These contributions offer a rigorous theoretical foundation for understanding model complexity in adaptive regression, bridging a critical gap between theory and practice.
翻译:惩罚回归模型的有效自由度量化了生成预测时实际使用的信息量,在模型评估与选择中起着关键作用。尽管Lasso惩罚存在闭式估计量,但包括自适应Lasso和自适应群组Lasso在内的常用惩罚方法的自适应扩展,始终缺乏类似的理论刻画。本文首次提出了这些方法有效自由度的无偏估计量及其主要理论性质,涵盖正交与非正交设计,这些结果均在Stein无偏风险估计框架下推导得出。所得表达式包含受正则化参数、系数符号和最小二乘估计影响的膨胀项。这些进展通过合成数据与真实数据展示了更精确的模型选择准则和无偏预测误差估计。本研究成果为理解自适应回归中的模型复杂度提供了严格的理论基础,弥合了理论与实践之间的关键鸿沟。