Deep neural networks are in the limelight of machine learning with their excellent performance in many data-driven applications. However, they can lead to inaccurate predictions when queried in out-of-distribution data points, which can have detrimental effects especially in sensitive domains, such as healthcare and transportation, where erroneous predictions can be very costly and/or dangerous. Subsequently, quantifying the uncertainty of the output of a neural network is often leveraged to evaluate the confidence of its predictions, and ensemble models have proved to be effective in measuring the uncertainty by utilizing the variance of predictions over a pool of models. In this paper, we propose a novel approach for uncertainty quantification via ensembles, called Random Activation Functions (RAFs) Ensemble, that aims at improving the ensemble diversity toward a more robust estimation, by accommodating each neural network with a different (random) activation function. Extensive empirical study demonstrates that RAFs Ensemble outperforms state-of-the-art ensemble uncertainty quantification methods on both synthetic and real-world datasets in a series of regression tasks.

翻译：深度神经网络凭借其在众多数据驱动应用中的卓越表现，已成为机器学习的焦点。然而，当在分布外数据点上进行查询时，它们可能导致不准确的预测，这尤其在医疗和交通等敏感领域可能产生有害影响——在这些领域中，错误预测的代价极其高昂且/或具有危险性。因此，通常利用神经网络输出不确定性的量化来评估其预测的置信度，而集成模型已被证明能够通过利用一组模型预测的方差来有效测量不确定性。本文提出了一种基于集成的不确定性量化新方法，称为随机激活函数集成，旨在通过为每个神经网络配备不同的（随机）激活函数来提高集成多样性，从而实现更鲁棒的估计。大量实证研究表明，在一系列回归任务中，RAFs集成在合成数据集和真实数据集上均优于最先进的集成不确定性量化方法。