Image classification with deep neural networks has reached state-of-art with high accuracy. This success is attributed to good internal representation features that bypasses the difficulties of the non-convex optimization problems. We have little understanding of these internal representations, let alone quantifying them. Recent research efforts have focused on alternative theories and explanations of the generalizability of these deep networks. We propose the alternative perturbation of deep models during their training induces changes that lead to transitions to different families. The result is an Anna Karenina Principle AKP for deep learning, in which less generalizable models unhappy families vary more in their representation than more generalizable models happy families paralleling Leo Tolstoy dictum that all happy families look alike, each unhappy family is unhappy in its own way. Anna Karenina principle has been found in systems in a wide range: from the surface of endangered corals exposed to harsh weather to the lungs of patients suffering from fatal diseases of AIDs. In our paper, we have generated artificial perturbations to our model by hot-swapping the activation and loss functions during the training. In this paper, we build a model to classify cancer cells from non-cancer ones. We give theoretical proof that the internal representations of generalizable happy models are similar in the asymptotic limit. Our experiments verify similar representations of generalizable models.

翻译：深度神经网络在图像分类任务中已达到最先进水平，准确率极高。这一成功归功于良好的内部表征特征，这些特征规避了非凸优化问题的困难。然而，我们对这些内部表征的理解仍然有限，更遑论对其进行量化。近年来的研究聚焦于解释这些深度网络泛化能力的替代理论。我们提出，在训练过程中对深度模型进行替代性扰动会引发变化，导致模型向不同族类转变。由此得出深度学习的安娜·卡列尼娜原理（AKP）：泛化能力较差的模型（不幸福的家庭）在表征上的差异大于泛化能力较强的模型（幸福的家庭），这与列夫·托尔斯泰的格言“所有幸福的家庭都相似，每个不幸的家庭各有各的不幸”相呼应。安娜·卡列尼娜原理已在多种系统中得到验证：从暴露于恶劣天气下的濒危珊瑚表面，到患有致命艾滋病患者的肺部。在我们的论文中，我们通过在训练过程中热切换激活函数和损失函数，对模型生成人工扰动。我们构建了一个模型用于区分癌细胞与非癌细胞。我们从理论上证明，泛化能力良好的（幸福）模型的内部表征在渐近极限下是相似的。我们的实验验证了泛化能力良好的模型具有相似的表征。