Graph Hypernetworks (GHN) can predict the parameters of varying unseen CNN architectures with surprisingly good accuracy at a fraction of the cost of iterative optimization. Following these successes, preliminary research has explored the use of GHNs to predict quantization-robust parameters for 8-bit and 4-bit quantized CNNs. However, this early work leveraged full-precision float32 training and only quantized for testing. We explore the impact of quantization-aware training and/or other quantization-based training strategies on quantized robustness and performance of GHN predicted parameters for low-precision CNNs. We show that quantization-aware training can significantly improve quantized accuracy for GHN predicted parameters of 4-bit quantized CNNs and even lead to greater-than-random accuracy for 2-bit quantized CNNs. These promising results open the door for future explorations such as investigating the use of GHN predicted parameters as initialization for further quantized training of individual CNNs, further exploration of "extreme bitwidth" quantization, and mixed precision quantization schemes.

翻译：图超网络（GHN）能够以惊人的准确度预测各种未见过的CNN架构参数，而计算成本仅为迭代优化的一小部分。基于这些成功，初步研究已探索利用GHN预测8位和4位量化CNN的量化鲁棒参数。然而，这类早期工作采用全精度float32训练，仅在进行测试时执行量化操作。本文探讨了量化感知训练和/或其他基于量化的训练策略对GHN预测的低精度CNN参数在量化鲁棒性与性能方面的影响。结果表明，量化感知训练可显著提升GHN预测参数在4位量化CNN上的量化准确度，甚至使2位量化CNN获得高于随机水平的准确率。这些突破性结果为后续研究打开了大门，例如探索将GHN预测参数作为单个CNN进一步量化训练的初始化参数、深入研究“极端位宽”量化方案以及混合精度量化策略。