Intelligence relies on an agent's knowledge of what it does not know. This capability can be assessed based on the quality of joint predictions of labels across multiple inputs. In principle, ensemble-based approaches produce effective joint predictions, but the computational costs of training large ensembles can become prohibitive. We introduce the epinet: an architecture that can supplement any conventional neural network, including large pretrained models, and can be trained with modest incremental computation to estimate uncertainty. With an epinet, conventional neural networks outperform very large ensembles, consisting of hundreds or more particles, with orders of magnitude less computation. The epinet does not fit the traditional framework of Bayesian neural networks. To accommodate development of approaches beyond BNNs, such as the epinet, we introduce the epistemic neural network (ENN) as an interface for models that produce joint predictions.

翻译：智力依赖于智能体对其未知知识的认知能力。这一能力可通过多输入场景下标签联合预测的质量进行评估。原则上，基于集成的方法能产生有效的联合预测，但训练大规模集成模型的计算成本可能高得难以承受。我们提出“epinet”：一种能够对任意传统神经网络（包括大型预训练模型）进行补充的架构，通过适度的增量计算即可训练以估计不确定性。使用epinet的传统神经网络在数量级更低的计算代价下，能够超越由数百甚至更多子模型组成的超大规模集成模型。Epinet并不符合贝叶斯神经网络的传统框架。为容纳诸如epinet等超越贝叶斯神经网络的方法，我们引入认知神经网络（ENN）作为生成联合预测模型的通用接口。