This study demonstrates that double descent can be mitigated by adding a dropout layer adjacent to the fully connected linear layer. The unexpected double-descent phenomenon garnered substantial attention in recent years, resulting in fluctuating prediction error rates as either sample size or model size increases. Our paper posits that the optimal test error, in terms of the dropout rate, shows a monotonic decrease in linear regression with increasing sample size. Although we do not provide a precise mathematical proof of this statement, we empirically validate through experiments that the test error decreases for each dropout rate. The statement we prove is that the expected test error for each dropout rate within a certain range decreases when the dropout rate is fixed. Our experimental results substantiate our claim, showing that dropout with an optimal dropout rate can yield a monotonic test error curve in nonlinear neural networks. These experiments were conducted using the Fashion-MNIST and CIFAR-10 datasets. These findings imply the potential benefit of incorporating dropout into risk curve scaling to address the peak phenomenon. To our knowledge, this study represents the first investigation into the relationship between dropout and double descent.

翻译：本研究证明了在紧邻全连接线性层处添加一个丢弃层可以缓解双重下降现象。近些年来，这种意外的双重下降现象引起了广泛关注，其表现为随着样本量或模型规模增大，预测错误率出现波动。本文指出，对于线性回归，以丢弃率为指标的最优测试误差随着样本量增加呈现单调下降趋势。虽然我们未能对此论断提供精确的数学证明，但通过实验验证了在每种丢弃率下测试误差确实下降。我们证明的命题是：当丢弃率固定时，在特定范围内每种丢弃率对应的期望测试误差均下降。我们的实验结果支持了这一论断，表明在非线性神经网络中，采用最优丢弃率的丢弃策略能够产生单调的测试误差曲线。这些实验是在Fashion-MNIST和CIFAR-10数据集上进行的。该发现意味着在风险曲线缩放中引入丢弃机制以应对峰值现象具有潜在价值。据我们所知，本研究首次探讨了丢弃与双重下降之间的关联。