Quantization is a promising approach to reduce the high computational complexity of image super-resolution (SR) networks. However, compared to high-level tasks like image classification, low-bit quantization leads to severe accuracy loss in SR networks. This is because feature distributions of SR networks are significantly divergent for each channel or input image, and is thus difficult to determine a quantization range. Existing SR quantization works approach this distribution mismatch problem by dynamically adapting quantization ranges to the variant distributions during test time. However, such dynamic adaptation incurs additional computational costs that limit the benefits of quantization. Instead, we propose a new quantization-aware training framework that effectively Overcomes the Distribution Mismatch problem in SR networks without the need for dynamic adaptation. Intuitively, the mismatch can be reduced by directly regularizing the variance in features during training. However, we observe that variance regularization can collide with the reconstruction loss during training and adversely impact SR accuracy. Thus, we avoid the conflict between two losses by regularizing the variance only when the gradients of variance regularization are cooperative with that of reconstruction. Additionally, to further reduce the distribution mismatch, we introduce distribution offsets to layers with a significant mismatch, which either scales or shifts channel-wise features. Our proposed algorithm, called ODM, effectively reduces the mismatch in distributions with minimal computational overhead. Experimental results show that ODM effectively outperforms existing SR quantization approaches with similar or fewer computations, demonstrating the importance of reducing the distribution mismatch problem. Our code is available at https://github.com/Cheeun/ODM.

翻译：量化是降低图像超分辨率（SR）网络高计算复杂度的一种有前景的方法。然而，与图像分类等高级任务相比，低位量化会导致SR网络严重的精度损失。这是因为SR网络的特征分布在每个通道或输入图像上存在显著差异，因此难以确定量化范围。现有的SR量化工作通过在测试时动态调整量化范围以适应变化的分布来处理这种分布不匹配问题。然而，这种动态适应会带来额外的计算成本，限制了量化的优势。相反，我们提出了一种新的量化感知训练框架，该框架能有效克服SR网络中的分布不匹配问题，而无需动态适应。直观上，通过在训练过程中直接正则化特征的方差可以减少不匹配。然而，我们观察到方差正则化可能与训练中的重建损失发生冲突，并对SR精度产生不利影响。因此，我们仅在方差正则化的梯度与重建梯度协作时进行正则化，从而避免这两个损失之间的冲突。此外，为了进一步减少分布不匹配，我们在存在显著不匹配的层中引入分布偏移，该偏移要么缩放要么平移通道级特征。我们提出的算法称为ODM，以最小的计算开销有效减少了分布中的不匹配。实验结果表明，ODM在相同或更少计算量下有效优于现有SR量化方法，证明了减少分布不匹配问题的重要性。我们的代码可在https://github.com/Cheeun/ODM获取。