Overparameterized deep networks that generalize well have been key to the dramatic success of deep learning in recent years. The reasons for their remarkable ability to generalize are not well understood yet. When class labels in the training set are shuffled to varying degrees, it is known that deep networks can still reach perfect training accuracy at the detriment of generalization to true labels -- a phenomenon that has been called memorization. It has, however, been unclear why the poor generalization to true labels that accompanies such memorization, comes about. One possibility is that during training, all layers of the network irretrievably re-organize their representations in a manner that makes generalization to true labels difficult. The other possibility is that one or more layers of the trained network retain significantly more latent ability to generalize to true labels, but the network somehow "chooses" to readout in a manner that is detrimental to generalization to true labels. Here, we provide evidence for the latter possibility by demonstrating, empirically, that such models possess information in their representations for substantially-improved generalization to true labels. Furthermore, such abilities can be easily decoded from the internals of the trained model, and we build a technique to do so. We demonstrate results on multiple models trained with standard datasets. Our code is available at: https://github.com/simranketha/MASC_DNN.

翻译：近年来，泛化性能优异的过参数化深度网络是深度学习取得巨大成功的关键。然而，其卓越泛化能力的原因尚未得到充分理解。已知当训练集中的类别标签被不同程度地打乱时，深度网络仍能达到完美的训练精度，但会损害对真实标签的泛化能力——这一现象被称为记忆效应。然而，为何伴随这种记忆效应会出现对真实标签的泛化能力下降，目前尚不清楚。一种可能性是：在训练过程中，网络的所有层都以不可逆的方式重组其表征，导致对真实标签的泛化变得困难。另一种可能性是：训练后的网络有一个或多个层保留了显著更强的对真实标签的泛化潜力，但网络以某种方式“选择”了不利于对真实标签泛化的读出机制。本文通过实证证明，此类模型在其表征中蕴含着可显著提升对真实标签泛化能力的信息，从而为后一种可能性提供了证据。此外，这种能力可以轻易地从训练模型的内部解码出来，我们构建了一种实现该解码的技术。我们在多个使用标准数据集训练的模型上展示了实验结果。代码发布于：https://github.com/simranketha/MASC_DNN。