Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of real-world systems constrain to smaller ensembles and lower-capacity networks, significantly deteriorating their performance and properties. We introduce Packed-Ensembles (PE), a strategy to design and train lightweight structured ensembles by carefully modulating the dimension of their encoding space. We leverage grouped convolutions to parallelize the ensemble into a single shared backbone and forward pass to improve training and inference speeds. PE is designed to operate within the memory limits of a standard neural network. Our extensive research indicates that PE accurately preserves the properties of DE, such as diversity, and performs equally well in terms of accuracy, calibration, out-of-distribution detection, and robustness to distribution shift. We make our code available at https://github.com/ENSTA-U2IS/torch-uncertainty.

翻译：深度集成（Deep Ensembles, DE）是实现精度、校准、不确定性估计和分布外检测等关键指标卓越性能的突出方法。然而，现实系统的硬件限制迫使使用较小的集成和低容量网络，从而显著降低其性能和性质。我们引入了打包集成（Packed-Ensembles, PE），这是一种通过精心调节其编码空间维度来设计和训练轻量级结构化集成的策略。我们利用分组卷积将集成并行化为单个共享主干和单次前向传播，以提高训练和推理速度。PE被设计为在标准神经网络的内存限制内运行。我们的广泛研究表明，PE精确地保留了DE的性质，如多样性，并在精度、校准、分布外检测以及对分布偏移的鲁棒性方面表现同样出色。我们在https://github.com/ENSTA-U2IS/torch-uncertainty提供我们的代码。