Model averaging has received much attention in the past two decades, which integrates available information by averaging over potential models. Although various model averaging methods have been developed, there are few literatures on the theoretical properties of model averaging from the perspective of stability, and the majority of these methods constrain model weights to a simplex. The aim of this paper is to introduce stability from statistical learning theory into model averaging. Thus, we define the stability, asymptotic empirical risk minimizer, generalization, and consistency of model averaging and study the relationship among them. Our results indicate that stability can ensure that model averaging has good generalization performance and consistency under reasonable conditions, where consistency means model averaging estimator can asymptotically minimize the mean squared prediction error. We also propose a L2-penalty model averaging method without limiting model weights and prove that it has stability and consistency. In order to reduce the impact of tuning parameter selection, we use 10-fold cross-validation to select a candidate set of tuning parameters and perform a weighted average of the estimators of model weights based on estimation errors. The Monte Carlo simulation and an illustrative application demonstrate the usefulness of the proposed method.

翻译：模型平均在过去二十年中备受关注，它通过对潜在模型进行平均来整合可用信息。尽管已有多种模型平均方法被提出，但从稳定性角度研究模型平均理论性质的文献仍较为匮乏，且大多数方法将模型权重约束在单纯形上。本文旨在将统计学习理论中的稳定性引入模型平均领域。为此，我们定义了模型平均的稳定性、渐近经验风险最小化、泛化能力与相合性，并探究了这些概念之间的内在关系。研究结果表明：在合理条件下，稳定性能够保证模型平均具有良好的泛化性能与相合性，其中相合性指模型平均估计量能渐近地最小化均方预测误差。我们还提出了一种不限制模型权重的L2-惩罚模型平均方法，并证明其具有稳定性与相合性。为降低调参参数选择的影响，我们采用10折交叉验证选取调参参数的候选集，并基于估计误差对模型权重估计量进行加权平均。蒙特卡洛模拟与实例应用验证了所提方法的有效性。