The one-epoch overfitting problem has drawn widespread attention, especially in CTR and CVR estimation models in search, advertising, and recommendation domains. These models which rely heavily on large-scale sparse categorical features, often suffer a significant decline in performance when trained for multiple epochs. Although recent studies have proposed heuristic solutions, they have not clearly identified the fundamental cause of this phenomenon. In this work, we provide a theoretical analysis that explains why overfitting occurs in models that use large-scale sparse categorical features. Based on this analysis, we propose an adaptive regularization method to address it. Our approach not only prevents the severe performance degradation observed during multi-epoch training, but also improves model performance within a single epoch. This method has already been deployed in online production systems.

翻译：单轮训练过拟合问题已引起广泛关注，尤其在搜索、广告和推荐领域的CTR与CVR预估模型中。这些模型高度依赖大规模稀疏类别特征，在进行多轮训练时经常出现性能显著下降。尽管近期研究提出了启发式解决方案，但尚未明确揭示该现象的根本成因。本文通过理论分析阐释了使用大规模稀疏类别特征的模型发生过拟合的原因，并基于此提出一种自适应正则化方法以应对该问题。我们的方法不仅避免了多轮训练中观察到的严重性能衰退，还能提升单轮训练内的模型性能。该方法已在在线生产系统中部署。