Modern predictive models are often deployed to environments in which computational budgets are dynamic. Anytime algorithms are well-suited to such environments as, at any point during computation, they can output a prediction whose quality is a function of computation time. Early-exit neural networks have garnered attention in the context of anytime computation due to their capability to provide intermediate predictions at various stages throughout the network. However, we demonstrate that current early-exit networks are not directly applicable to anytime settings, as the quality of predictions for individual data points is not guaranteed to improve with longer computation. To address this shortcoming, we propose an elegant post-hoc modification, based on the Product-of-Experts, that encourages an early-exit network to become gradually confident. This gives our deep models the property of conditional monotonicity in the prediction quality -- an essential stepping stone towards truly anytime predictive modeling using early-exit architectures. Our empirical results on standard image-classification tasks demonstrate that such behaviors can be achieved while preserving competitive accuracy on average.

翻译：现代预测模型常被部署于计算预算动态变化的环境中。任意时间算法因其能在计算过程的任意时刻输出与计算时间相关的预测结果，非常适合此类场景。早期退出神经网络凭借在网络各阶段提供中间预测的能力，在任意时间计算领域备受关注。然而，我们证明现有早期退出网络并不直接适用于任意时间场景——个体数据点的预测质量未必随计算时间延长而提升。针对这一缺陷，我们提出一种基于专家乘积的优雅后验修正方法，促使早期退出网络逐步建立置信度。该方法赋予深度模型预测质量的条件单调性特征，这是利用早期退出架构实现真正任意时间预测建模的关键基石。在标准图像分类任务上的实证结果表明，该方法能在保持平均竞争力精度的同时实现上述行为特性。