Convolutional Neural Networks (CNNs) are the predominant model used for a variety of medical image analysis tasks. At inference time, these models are computationally intensive, especially with volumetric data. In principle, it is possible to trade accuracy for computational efficiency by manipulating the rescaling factor in the downsample and upsample layers of CNN architectures. However, properly exploring the accuracy-efficiency trade-off is prohibitively expensive with existing models. To address this, we introduce Scale-Space HyperNetworks (SSHN), a method that learns a spectrum of CNNs with varying internal rescaling factors. A single SSHN characterizes an entire Pareto accuracy-efficiency curve of models that match, and occasionally surpass, the outcomes of training many separate networks with fixed rescaling factors. We demonstrate the proposed approach in several medical image analysis applications, comparing SSHN against strategies with both fixed and dynamic rescaling factors. We find that SSHN consistently provides a better accuracy-efficiency trade-off at a fraction of the training cost. Trained SSHNs enable the user to quickly choose a rescaling factor that appropriately balances accuracy and computational efficiency for their particular needs at inference.

翻译：卷积神经网络（CNN）是目前广泛应用于多种医学图像分析任务的主流模型。在推理阶段，这些模型计算量巨大，处理三维数据时尤为突出。原则上，通过调整CNN架构中下采样层和上采样层的缩放因子，可以实现精度与计算效率的权衡。然而，使用现有模型适当探索精度-效率的权衡关系会带来难以承受的计算成本。为解决这一问题，我们提出了尺度空间超网络（SSHN），该方法能够学习一系列具有不同内部缩放因子的CNN网络。单个SSHN即可表征完整的Pareto精度-效率模型曲线，其性能可与甚至有时超越分别训练多个固定缩放因子网络的方案。我们在多项医学图像分析应用中验证了所提方法，并将SSHN与固定缩放因子和动态缩放因子的策略进行对比。结果表明，SSHN能以更低的训练成本持续提供更优的精度-效率权衡。训练完成的SSHN使用户能够在推理阶段快速选择适合特定需求的缩放因子，从而恰当平衡精度与计算效率。