Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network. However, these methods still require multiple forward passes for prediction, leading to a significant computational cost. In this work, we show a surprising result: the benefits of using multiple predictions can be achieved `for free' under a single model's forward pass. In particular, we show that, using a multi-input multi-output (MIMO) configuration, one can utilize a single model's capacity to train multiple subnetworks that independently learn the task at hand. By ensembling the predictions made by the subnetworks, we improve model robustness without increasing compute. We observe a significant improvement in negative log-likelihood, accuracy, and calibration error on CIFAR10, CIFAR100, ImageNet, and their out-of-distribution variants compared to previous methods.

翻译：最近对高效混合神经网络采取的办法表明,在原始网络参数上少有增益,就能够实现强大的稳健性和不确定性的性能,然而,这些方法仍然需要多个前方的预测,从而导致巨大的计算成本。在这项工作中,我们显示出一个令人惊讶的结果:在单一模型的前方传球下,“免费”使用多种预测的好处是可以实现的。特别是,我们表明,使用多投入多产出(MIMO)配置,可以使用单一模型的能力来培训独立学习手头任务的许多子网络。通过结合子网络所作的预测,我们提高模型的稳健性,而不增加计算。我们观察到,与以往方法相比,CIFAR10、CIFAR100、图像网的负日志、准确度和校准错误及其外分配变体有了很大的改进。