The phenomenon of double descent has recently gained attention in supervised learning. It challenges the conventional wisdom of the bias-variance trade-off by showcasing a surprising behavior. As the complexity of the model increases, the test error initially decreases until reaching a certain point where the model starts to overfit the train set, causing the test error to rise. However, deviating from classical theory, the error exhibits another decline when exceeding a certain degree of over-parameterization. We study the presence of double descent in unsupervised learning, an area that has received little attention and is not yet fully understood. We conduct extensive experiments using under-complete auto-encoders (AEs) for various applications, such as dealing with noisy data, domain shifts, and anomalies. We use synthetic and real data and identify model-wise, epoch-wise, and sample-wise double descent for all the aforementioned applications. Finally, we assessed the usability of the AEs for detecting anomalies and mitigating the domain shift between datasets. Our findings indicate that over-parameterized models can improve performance not only in terms of reconstruction, but also in enhancing capabilities for the downstream task.

翻译：双重下降现象近期在监督学习中引起了广泛关注。该现象通过展示出人意料的行为，对传统的偏差-方差权衡观念提出了挑战。随着模型复杂度的增加，测试误差首先下降直至某临界点，此后模型开始对训练集过拟合，导致测试误差上升。然而与经典理论不同，当超过特定程度的过参数化后，误差会呈现再次下降的趋势。本文研究双重下降现象在无监督学习中的存在性，该领域目前关注较少且尚未被完全理解。我们通过欠完备自编码器（AEs）在多种应用场景（如噪声数据处理、领域偏移和异常检测）中进行了大量实验。基于合成数据与真实数据，我们在所有前述应用中都观察到了模型维度、训练轮次维度和样本维度的双重下降现象。最后，我们评估了自编码器在异常检测和缓解数据集间领域偏移方面的实用性。研究结果表明，过参数化模型不仅能提升重构性能，还能增强下游任务的处理能力。