Modern predictive models are often deployed to environments in which computational budgets are dynamic. Anytime algorithms are well-suited to such environments as, at any point during computation, they can output a prediction whose quality is a function of computation time. Early-exit neural networks have garnered attention in the context of anytime computation due to their capability to provide intermediate predictions at various stages throughout the network. However, we demonstrate that current early-exit networks are not directly applicable to anytime settings, as the quality of predictions for individual data points is not guaranteed to improve with longer computation. To address this shortcoming, we propose an elegant post-hoc modification, based on the Product-of-Experts, that encourages an early-exit network to become gradually confident. This gives our deep models the property of conditional monotonicity in the prediction quality -- an essential stepping stone towards truly anytime predictive modeling using early-exit architectures. Our empirical results on standard image-classification tasks demonstrate that such behaviors can be achieved while preserving competitive accuracy on average.

翻译：现代预测模型常被部署到计算预算动态变化的环境中。任意时刻算法非常适合此类环境，因为其在计算过程中的任何时刻都能输出预测结果，且预测质量是计算时间的函数。早期退出神经网络因其能在网络各阶段提供中间预测的能力，在任意时刻计算领域备受关注。然而，我们证明现有早期退出网络无法直接应用于任意时刻场景，因为单个数据点的预测质量并不保证随计算时间延长而提升。为解决这一缺陷，我们提出一种基于专家乘积的优雅后验修改方案，促使早期退出网络逐步建立置信度。这使我们的深度模型获得了预测质量的条件单调性特性——这是利用早期退出架构实现真正任意时刻预测建模的关键基石。我们在标准图像分类任务上的实验结果表明，在保持具有竞争力的平均精度的同时，可以实现此类行为。