Recent works have demonstrated a double descent phenomenon in over-parameterized learning. Although this phenomenon has been investigated by recent works, it has not been fully understood in theory. In this paper, we investigate the multiple descent phenomenon in a class of multi-component prediction models. We first consider a ''double random feature model'' (DRFM) concatenating two types of random features, and study the excess risk achieved by the DRFM in ridge regression. We calculate the precise limit of the excess risk under the high dimensional framework where the training sample size, the dimension of data, and the dimension of random features tend to infinity proportionally. Based on the calculation, we further theoretically demonstrate that the risk curves of DRFMs can exhibit triple descent. We then provide a thorough experimental study to verify our theory. At last, we extend our study to the ''multiple random feature model'' (MRFM), and show that MRFMs ensembling $K$ types of random features may exhibit $(K+1)$-fold descent. Our analysis points out that risk curves with a specific number of descent generally exist in learning multi-component prediction models.

翻译：近期研究表明，过参数化学习存在双重下降现象。尽管该现象已被多项研究探讨，但其理论机制尚未完全明晰。本文针对一类多组件预测模型中的多重下降现象展开研究。我们首先考虑拼接两类随机特征的"双随机特征模型"，研究岭回归中该模型产生的超额风险。在高维框架下（训练样本量、数据维度和随机特征维度按比例趋于无穷），我们精确计算出超额风险的极限值。基于此计算，我们从理论上证明双随机特征模型的风险曲线可呈现三重下降特征。随后通过详尽的实验研究验证理论结果。最后将研究拓展至"多重随机特征模型"，证明集成K类随机特征的MRFM可呈现(K+1)重下降现象。我们的分析表明，具有特定下降次数的风险曲线普遍存在于多组件预测模型的学习过程中。