The rising performance of deep neural networks is often empirically attributed to an increase in the available computational power, which allows complex models to be trained upon large amounts of annotated data. However, increased model complexity leads to costly deployment of modern neural networks, while gathering such amounts of data requires huge costs to avoid label noise. In this work, we study the ability of compression methods to tackle both of these problems at once. We hypothesize that quantization-aware training, by restricting the expressivity of neural networks, behaves as a regularization. Thus, it may help fighting overfitting on noisy data while also allowing for the compression of the model at inference. We first validate this claim on a controlled test with manually introduced label noise. Furthermore, we also test the proposed method on Facial Action Unit detection, where labels are typically noisy due to the subtlety of the task. In all cases, our results suggests that quantization significantly improve the results compared with existing baselines, regularization as well as other compression methods.

翻译：深度神经网络性能的提升通常被经验性地归因于可用计算能力的增强，这使得复杂模型能够在大规模标注数据上训练。然而，模型复杂度的增加导致现代神经网络部署成本高昂，而收集如此规模的数据又需付出巨大代价以避免标签噪声。本研究探讨了压缩方法同时解决上述两个问题的能力。我们假设量化感知训练通过限制神经网络的表达能力起到正则化作用，因而有助于抑制对噪声数据的过拟合，同时允许在推理阶段实现模型压缩。我们首先在人为引入标签噪声的受控实验中验证了这一观点，此外，还将所提方法应用于面部动作单元检测任务——该任务因精细化程度高而存在典型标签噪声。所有实验结果表明，与现有基线方法、正则化技术及其他压缩方法相比，量化显著提升了模型性能。