In the field of neural data compression, the prevailing focus has been on optimizing algorithms for either classical distortion metrics, such as PSNR or SSIM, or human perceptual quality. With increasing amounts of data consumed by machines rather than humans, a new paradigm of machine-oriented compression$\unicode{x2013}$which prioritizes the retention of features salient for machine perception over traditional human-centric criteria$\unicode{x2013}$has emerged, creating several new challenges to the development, evaluation, and deployment of systems utilizing lossy compression. In particular, it is unclear how different approaches to lossy compression will affect the performance of downstream machine perception tasks. To address this under-explored area, we evaluate various perception models$\unicode{x2013}$including image classification, image segmentation, speech recognition, and music source separation$\unicode{x2013}$under severe lossy compression. We utilize several popular codecs spanning conventional, neural, and generative compression architectures. Our results indicate three key findings: (1) using generative compression, it is feasible to leverage highly compressed data while incurring a negligible impact on machine perceptual quality; (2) machine perceptual quality correlates strongly with deep similarity metrics, indicating a crucial role of these metrics in the development of machine-oriented codecs; and (3) using lossy compressed datasets, (e.g. ImageNet) for pre-training can lead to counter-intuitive scenarios where lossy compression increases machine perceptual quality rather than degrading it. To encourage engagement on this growing area of research, our code and experiments are available at: https://github.com/danjacobellis/MPQ.
翻译:在神经数据压缩领域,现有研究主要集中于针对经典失真度量(如PSNR或SSIM)或人类感知质量优化算法。随着机器而非人类消费的数据量日益增长,一种以"面向机器的压缩"为特征的新范式已出现——该范式优先保留对机器感知起关键作用的特征,而非沿用传统以人为中心的准则。这一转变为利用有损压缩系统的开发、评估与部署带来了若干新挑战。特别值得注意的是,不同有损压缩方法将如何影响下游机器感知任务的性能尚不明确。为探索这一未充分研究的领域,我们在严重有损压缩条件下评估了包括图像分类、图像分割、语音识别及音乐源分离在内的多种感知模型,并采用涵盖传统、神经及生成式压缩架构的多种主流编解码器。研究结果揭示了三个关键发现:(1)利用生成式压缩技术,可在对机器感知质量影响极小的情况下使用高度压缩的数据;(2)机器感知质量与深度相似度度量存在强相关性,表明这些度量在面向机器编解码器的开发中具有关键作用;(3)使用有损压缩数据集(如ImageNet)进行预训练可能导致反直觉现象——有损压缩反而提升而非降低机器感知质量。为促进该新兴研究领域的深入探讨,相关代码与实验数据可通过 https://github.com/danjacobellis/MPQ 获取。