Deep neural network ensembles hold the potential of improving generalization performance for complex learning tasks. This paper presents formal analysis and empirical evaluation to show that heterogeneous deep ensembles with high ensemble diversity can effectively leverage model learning heterogeneity to boost ensemble robustness. We first show that heterogeneous DNN models trained for solving the same learning problem, e.g., object detection, can significantly strengthen the mean average precision (mAP) through our weighted bounding box ensemble consensus method. Second, we further compose ensembles of heterogeneous models for solving different learning problems, e.g., object detection and semantic segmentation, by introducing the connected component labeling (CCL) based alignment. We show that this two-tier heterogeneity driven ensemble construction method can compose an ensemble team that promotes high ensemble diversity and low negative correlation among member models of the ensemble, strengthening ensemble robustness against both negative examples and adversarial attacks. Third, we provide a formal analysis of the ensemble robustness in terms of negative correlation. Extensive experiments validate the enhanced robustness of heterogeneous ensembles in both benign and adversarial settings. The source codes are available on GitHub at https://github.com/git-disl/HeteRobust.

翻译：深度神经网络集成有望提升复杂学习任务的泛化性能。本文通过形式化分析与实验评估表明，具有高集成异质性的异构深度集成能够有效利用模型学习异质性来增强集成鲁棒性。首先，我们证明为解决同一学习问题（如目标检测）而训练的异构DNN模型，通过我们提出的加权边界框集成共识方法可显著提升平均精度（mAP）。其次，通过引入基于连通分量标记（CCL）的对齐方法，我们进一步构建了解决不同学习问题（如目标检测与语义分割）的异构模型集成。研究表明，这种双层异质性驱动的集成构建方法能够组成一个集成团队，该团队促进集成成员模型间的高异质性与低负相关性，从而增强集成对抗负样本与对抗攻击的鲁棒性。第三，我们从负相关性角度对集成鲁棒性进行了形式化分析。大量实验验证了异构集成在良性环境与对抗环境下的增强鲁棒性。源代码已发布于GitHub：https://github.com/git-disl/HeteRobust。