JPEG, a lossy image compression technique designed for human viewers, has maintained its dominance for decades. However, in the era of artificial intelligence (AI), a substantial portion of image data, often compressed by JPEG, is and will continue to be consumed by deep neural networks (DNNs) instead of humans, thus creating a need to optimize JPEG for DNN inference performance. To this end, we propose learned JPEG compression for DNN vision (J4D), a novel training framework for determining JPEG encoding parameters to minimize compression rate while maximizing DNN inference performance. The major challenge of solving this optimization problem lies in representing the JPEG codec and compression rate in closed form. By incorporating a differentiable soft quantizer based on a probabilistic quantization scheme, we not only obtain a differentiable proxy for the JPEG codec, but are also able to compute the entropy of the coded source analytically, which is a close estimate of the actual compression rate. Equipped with both the differentiable JPEG codec and the information-theoretic rate estimator, we are then able to solve the aforementioned optimization problem with backpropagation. After training, the learned encoding parameters will be subsequently used in actual JPEG encoding based on probabilistic quantization. Extensive experimental results across multiple datasets and DNN architectures demonstrate that J4D consistently and significantly outperforms the default JPEG and other competitive JPEG codecs optimized for DNNs. Notably, compared to the default JPEG, J4D achieves an increase in accuracy by as much as 11.60% at the same rate, or a reduction of compression rate up to 80.05% at the same accuracy. Additionally, with the help of J4D, we show the potential to design universal JPEG encoding parameters for various DNN architectures for the first time.

翻译：摘要：JPEG作为一种专为人类视觉设计的有损图像压缩技术，数十年来一直占据主导地位。然而在人工智能时代，大量经JPEG压缩的图像数据正在并将持续被深度神经网络而非人类所消费，因此需要针对深度神经网络推理性能优化JPEG。为此，我们提出面向深度神经网络视觉的学习型JPEG压缩（J4D）——一种新型训练框架，用于确定JPEG编码参数，在最小化压缩率的同时最大化深度神经网络推理性能。解决该优化问题的主要挑战在于如何以闭合形式表示JPEG编解码器及压缩率。通过引入基于概率量化方案的可微软量化器，我们不仅获得了JPEG编解码器的可微代理，还能解析计算编码源的熵，该熵值能紧密估计实际压缩率。借助可微JPEG编解码器与信息论速率估计器，我们即可通过反向传播求解上述优化问题。训练完成后，学习到的编码参数将用于基于概率量化的实际JPEG编码。跨多个数据集和深度神经网络架构的大量实验表明，J4D始终显著优于默认JPEG及其他面向深度神经网络优化的竞争性JPEG编解码器。值得注意的是，与默认JPEG相比，J4D在同码率下准确率提升高达11.60%，或在同等准确率下压缩率降低高达80.05%。此外，借助J4D，我们首次展示了为不同深度神经网络架构设计通用JPEG编码参数的潜力。