There is a growing interest in model-based deep learning (MBDL) for solving imaging inverse problems. MBDL networks can be seen as iterative algorithms that estimate the desired image using a physical measurement model and a learned image prior specified using a convolutional neural net (CNNs). The iterative nature of MBDL networks increases the test-time computational complexity, which limits their applicability in certain large-scale applications. We address this issue by presenting structured pruning algorithm for model-based deep learning (SPADE) as the first structured pruning algorithm for MBDL networks. SPADE reduces the computational complexity of CNNs used within MBDL networks by pruning its non-essential weights. We propose three distinct strategies to fine-tune the pruned MBDL networks to minimize the performance loss. Each fine-tuning strategy has a unique benefit that depends on the presence of a pre-trained model and a high-quality ground truth. We validate SPADE on two distinct inverse problems, namely compressed sensing MRI and image super-resolution. Our results highlight that MBDL models pruned by SPADE can achieve substantial speed up in testing time while maintaining competitive performance.

翻译：基于模型的深度学习（MBDL）在解决成像逆问题方面日益受到关注。MBDL网络可视为迭代算法，其通过物理测量模型和使用卷积神经网络（CNN）学习的图像先验知识来估计目标图像。MBDL网络的迭代特性增加了测试时的计算复杂度，限制了其在大规模应用中的适用性。为解决这一问题，我们提出基于模型的深度学习结构化剪枝算法（SPADE），这是首个针对MBDL网络的结构化剪枝算法。SPADE通过剪枝非必要权重，降低了MBDL网络中CNN的计算复杂度。我们提出了三种不同的微调策略来优化剪枝后的MBDL网络，以最小化性能损失。每种微调策略具有独特优势，其有效性取决于预训练模型和高质量真实数据的存在性。我们在压缩感知磁共振成像和图像超分辨率这两个不同的逆问题上验证了SPADE的性能。结果表明，经SPADE剪枝的MBDL模型在保持竞争性性能的同时，可显著提升测试阶段的运行速度。