Accurate uncertainty estimation is crucial for deploying neural networks in risk-sensitive applications such as medical diagnosis. Monte Carlo Dropout is a widely used technique for approximating predictive uncertainty by performing stochastic forward passes with dropout during inference. However, using static dropout rates across all layers and inputs can lead to suboptimal uncertainty estimates, as it fails to adapt to the varying characteristics of individual inputs and network layers. Existing approaches optimize dropout rates during training using labeled data, resulting in fixed inference-time parameters that cannot adjust to new data distributions, compromising uncertainty estimates in Monte Carlo simulations. In this paper, we propose Rate-In, an algorithm that dynamically adjusts dropout rates during inference by quantifying the information loss induced by dropout in each layer's feature maps. By treating dropout as controlled noise injection and leveraging information-theoretic principles, Rate-In adapts dropout rates per layer and per input instance without requiring ground truth labels. By quantifying the functional information loss in feature maps, we adaptively tune dropout rates to maintain perceptual quality across diverse medical imaging tasks and architectural configurations. Our extensive empirical study on synthetic data and real-world medical imaging tasks demonstrates that Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates without compromising predictive performance. Rate-In offers a practical, unsupervised, inference-time approach to optimizing dropout for more reliable predictive uncertainty estimation in critical applications.

翻译：准确的不确定性估计对于在医疗诊断等风险敏感应用中部署神经网络至关重要。蒙特卡洛丢弃是一种广泛使用的技术，通过在推理时执行带有丢弃的随机前向传播来近似预测不确定性。然而，在所有层和输入上使用静态丢弃率可能导致次优的不确定性估计，因为它无法适应不同输入和网络层的特性变化。现有方法利用标注数据在训练期间优化丢弃率，导致推理时参数固定，无法适应新的数据分布，从而损害了蒙特卡洛模拟中的不确定性估计。本文提出Rate-In算法，该算法通过量化丢弃操作在每层特征图中引起的信息损失，在推理时动态调整各层丢弃率。通过将丢弃视为受控噪声注入并利用信息论原理，Rate-In能够针对每个输入实例逐层自适应调整丢弃率，且无需真实标签。通过量化特征图中的功能性信息损失，我们自适应调整丢弃率以在多样化的医学成像任务和架构配置中保持感知质量。我们在合成数据和真实世界医学成像任务上的大量实证研究表明，与固定或启发式丢弃率相比，Rate-In在保持预测性能的同时，显著改善了校准效果并锐化了不确定性估计。Rate-In为关键应用中实现更可靠的预测不确定性估计，提供了一种实用、无监督的推理时丢弃优化方法。