Quantization has emerged as a promising direction for model compression. Recently, data-free quantization has been widely studied as a promising method to avoid privacy concerns, which synthesizes images as an alternative to real training data. Existing methods use classification loss to ensure the reliability of the synthesized images. Unfortunately, even if these images are well-classified by the pre-trained model, they still suffer from low semantics and homogenization issues. Intuitively, these low-semantic images are sensitive to perturbations, and the pre-trained model tends to have inconsistent output when the generator synthesizes an image with poor semantics. To this end, we propose Robustness-Guided Image Synthesis (RIS), a simple but effective method to enrich the semantics of synthetic images and improve image diversity, further boosting the performance of downstream data-free compression tasks. Concretely, we first introduce perturbations on input and model weight, then define the inconsistency metrics at feature and prediction levels before and after perturbations. On the basis of inconsistency on two levels, we design a robustness optimization objective to enhance the semantics of synthetic images. Moreover, we also make our approach diversity-aware by forcing the generator to synthesize images with small correlations in the label space. With RIS, we achieve state-of-the-art performance for various settings on data-free quantization and can be extended to other data-free compression tasks.

翻译：量化已成为模型压缩的一个有前景的方向。近来，无数据量化作为一种避免隐私问题的有效方法被广泛研究，该方法通过合成图像来替代真实训练数据。现有方法使用分类损失来确保合成图像的可靠性。不幸的是，即使这些图像能被预训练模型准确分类，它们仍然面临语义层级低和同质化问题。直观来看，这些低语义图像对扰动敏感，且当生成器合成语义较差的图像时，预训练模型往往会输出不一致的结果。为此，我们提出鲁棒引导的图像合成（RIS）——一种简单但有效的方法，用以丰富合成图像的语义并提升多样性，从而进一步促进下游无数据压缩任务的性能。具体而言，我们首先在输入和模型权重上引入扰动，然后定义扰动前后特征层面和预测层面的不一致性指标。基于两个层面的不一致性，我们设计了一个鲁棒优化目标来增强合成图像的语义。此外，我们迫使生成器合成标签空间相关性较小的图像，使方法具备多样性感知能力。通过RIS，我们在无数据量化的多种设置下实现了最先进的性能，并且该方法可扩展至其他无数据压缩任务。