Quantization has become a mainstream compression technique for reducing model size, computational requirements, and energy consumption for modern deep neural networks (DNNs). With the improved numerical support in recent hardware, including multiple variants of integer and floating point, mixed-precision quantization has become necessary to achieve high-quality results with low model cost. Prior mixed-precision quantization methods have performed a post-training quantization search, which compromises on accuracy, or a differentiable quantization search, which leads to high memory usage from branching. Therefore, we propose the first one-shot mixed-precision quantization search that eliminates the need for retraining in both integer and low-precision floating point models. We evaluate our floating-point and integer quantization search (FLIQS) on multiple convolutional networks and vision transformer models to discover Pareto-optimal models. Our approach discovers models that improve upon uniform precision, manual mixed-precision, and recent integer quantization search methods. With the proposed integer quantization search, we increase the accuracy of ResNet-18 on ImageNet by 1.31% points and ResNet-50 by 0.90% points with equivalent model cost over previous methods. Additionally, for the first time, we explore a novel mixed-precision floating-point search and improve MobileNetV2 by up to 0.98% points compared to prior state-of-the-art FP8 models. Finally, we extend FLIQS to simultaneously search a joint quantization and neural architecture space and improve the ImageNet accuracy by 2.69% points with similar model cost on a MobileNetV2 search space.

翻译：量化已成为压缩现代深度神经网络（DNN）模型规模、计算需求和能耗的主流技术。随着近期硬件对数值计算支持的改进（包括整数和浮点数的多种变体），混合精度量化对于在以低模型成本实现高质量结果的过程中变得不可或缺。以往的混合精度量化方法要么采用后训练量化搜索（会牺牲精度），要么使用可微量化搜索（因分支结构导致内存占用过高）。因此，我们提出首个一次性混合精度量化搜索方法，无需对整数和低精度浮点模型进行重新训练。我们在多个卷积网络和视觉Transformer模型上评估了所提出的浮点与整数量化搜索方法（FLIQS），以发现帕累托最优模型。该方法发现的模型性能优于均匀精度、手动混合精度以及近期整数量化搜索方法。通过所提出的整数量化搜索，我们在ImageNet数据集上使ResNet-18的准确率提升1.31个百分点，ResNet-50提升0.90个百分点（在模型成本与先前方法相当的前提下）。此外，我们首次探索了新型混合精度浮点搜索，在MobileNetV2上相比先前最优的FP8模型提升了高达0.98个百分点的准确率。最后，我们将FLIQS扩展至联合搜索量化与神经架构空间，在MobileNetV2搜索空间中以相近模型成本将ImageNet准确率提升2.69个百分点。