A fundamental question in designing lossy data compression schemes is how well one can do in comparison with the rate-distortion function, which describes the known theoretical limits of lossy compression. Motivated by the empirical success of deep neural network (DNN) compressors on large, real-world data, we investigate methods to estimate the rate-distortion function on such data, which would allow comparison of DNN compressors with optimality. While one could use the empirical distribution of the data and apply the Blahut-Arimoto algorithm, this approach presents several computational challenges and inaccuracies when the datasets are large and high-dimensional, such as the case of modern image datasets. Instead, we re-formulate the rate-distortion objective, and solve the resulting functional optimization problem using neural networks. We apply the resulting rate-distortion estimator, called NERD, on popular image datasets, and provide evidence that NERD can accurately estimate the rate-distortion function. Using our estimate, we show that the rate-distortion achievable by DNN compressors are within several bits of the rate-distortion function for real-world datasets. Additionally, NERD provides access to the rate-distortion achieving channel, as well as samples from its output marginal. Therefore, using recent results in reverse channel coding, we describe how NERD can be used to construct an operational one-shot lossy compression scheme with guarantees on the achievable rate and distortion. Experimental results demonstrate competitive performance with DNN compressors.

翻译：在构建有损数据压缩方案时，一个基本问题在于与描述有损压缩已知理论极限的率失真函数相比较，能达到多高的性能。受深度神经网络压缩器在大型真实世界数据上取得经验成功的启发，我们研究了在此类数据上估计率失真函数的方法，这将有助于将深度神经网络压缩器与最优性进行比较。虽然可以使用数据的经验分布并应用Blahut-Arimoto算法，但当数据集规模大且维度高时（例如现代图像数据集），该方法面临计算挑战且精度不足。为此，我们重新表述了率失真目标，并利用神经网络求解由此产生的函数优化问题。我们将所得到的率失真估计器命名为NERD，并在常见图像数据集上进行应用，提供了NERD能准确估计率失真函数的证据。利用我们的估计，我们证明深度神经网络压缩器所能达到的率失真与实际数据集的率失真函数仅相差数个比特。此外，NERD还能获取达到率失真的信道及其输出边缘分布样本。因此，利用反向信道编码的最新成果，我们描述了如何利用NERD构造一个在可达速率和失真方面有保证的操作型单次有损压缩方案。实验结果表明，该方案与深度神经网络压缩器相比具有竞争力。