Quantization has become a mainstream compression technique for reducing model size, computational requirements, and energy consumption for modern deep neural networks (DNNs). With improved numerical support in recent hardware, including multiple variants of integer and floating point, mixed-precision quantization has become necessary to achieve high-quality results with low model cost. Prior mixed-precision methods have performed either a post-training quantization search, which compromises on accuracy, or a differentiable quantization search, which leads to high memory usage from branching. Therefore, we propose the first one-shot mixed-precision quantization search that eliminates the need for retraining in both integer and low-precision floating point models. We evaluate our search (FLIQS) on multiple convolutional and vision transformer networks to discover Pareto-optimal models. Our approach improves upon uniform precision, manual mixed-precision, and recent integer quantization search methods. With integer models, we increase the accuracy of ResNet-18 on ImageNet by 1.31% and ResNet-50 by 0.90% with equivalent model cost over previous methods. Additionally, for the first time, we explore a novel mixed-precision floating-point search and improve MobileNetV2 by up to 0.98% compared to prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously search a joint quantization and neural architecture space and improve the ImageNet accuracy by 2.69% with similar model cost on a MobileNetV2 search space.

翻译：量化已成为降低现代深度神经网络（DNNs）模型规模、计算需求和能耗的主流压缩技术。随着近期硬件对数值的支持增强（包括整数和浮点的多种变体），混合精度量化对于以低模型成本实现高质量结果变得不可或缺。以往混合精度方法要么采用后训练量化搜索（牺牲精度），要么采用可微量化搜索（因分支导致高内存占用）。为此，我们首次提出一种无需重新训练的一次性混合精度量化搜索方法，可同时适用于整数和低精度浮点模型。我们在多个卷积网络和视觉Transformer网络上评估该搜索方法（FLIQS），以发现帕累托最优模型。我们的方法在均匀精度、手工混合精度及近期整数量化搜索方法上均有改进。在整数模型中，以等价模型成本相比先前方法，ResNet-18在ImageNet上的精度提升1.31%，ResNet-50提升0.90%。此外，我们首次探索了一种新型混合精度浮点搜索，在MobileNetV2上较之前最优FP8模型提升高达0.98%。最后，我们将FLIQS扩展至联合搜索量化与神经架构空间，在MobileNetV2搜索空间中以相似模型成本将ImageNet精度提升2.69%。