Deep neural network (DNN) with dropout can be regarded as an ensemble model consisting of lots of sub-DNNs (i.e., an ensemble sub-DNN where the sub-DNN is the remaining part of the DNN after dropout), and through increasing the diversity of the ensemble sub-DNN, the generalization and robustness of the DNN can be effectively improved. In this paper, a mask-guided divergence loss function (MDL), which consists of a cross-entropy loss term and an orthogonal term, is proposed to increase the diversity of the ensemble sub-DNN by the added orthogonal term. Particularly, the mask technique is introduced to assist in generating the orthogonal term for avoiding overfitting of the diversity learning. The theoretical analysis and extensive experiments on 4 datasets (i.e., MNIST, FashionMNIST, CIFAR10, and CIFAR100) manifest that MDL can improve the generalization and robustness of standard training and adversarial training. For CIFAR10 and CIFAR100, in standard training, the maximum improvement of accuracy is $1.38\%$ on natural data, $30.97\%$ on FGSM (i.e., Fast Gradient Sign Method) attack, $38.18\%$ on PGD (i.e., Projected Gradient Descent) attack. While in adversarial training, the maximum improvement is $1.68\%$ on natural data, $4.03\%$ on FGSM attack and $2.65\%$ on PGD attack.

翻译：深度神经网络(DNN)中包含辍学的深神经网络(DNN)可被视为由许多次DNN(即混合子DNN)组成的混合模型(即混合子DNN是辍学后DNN的剩余部分),并通过提高混合子DNN的多样性,可以有效地改进DNN的概括性和稳健性。在本文中,由跨轨损失期和正数术语组成的蒙面引导差异损失函数(MDL),提议通过添加正数术语来增加混合子DNNN(即混合子DNNN是DN的剩余部分)的多样化。尤其是,采用掩罩技术协助生成交错词,以避免过度适应多样性学习。在4个数据集(即MNIST、FAshion MNIST、CIFAR10和CIFAR100)中显示,ML可以改进标准培训和对抗性培训的普及性和稳健性。对于CFFFAR10和CIFAR100美元袭击的升级方法而言,关于SOAL Ralendalalalalalal 数据,关于PAnialalalal dreal dreal dreal drealal dreal dreal drealalal drealal.