One-shot channel simulation has recently emerged as a promising alternative to quantization and entropy coding in machine-learning-based lossy data compression schemes. However, while there are several potential applications of channel simulation - lossy compression with realism constraints or differential privacy, to name a few - little is known about its fundamental limitations. In this paper, we restrict our attention to a subclass of channel simulation protocols called causal rejection samplers (CRS), establish new, tighter lower bounds on their expected runtime and codelength, and demonstrate the bounds' achievability. Concretely, for an arbitrary CRS, let $Q$ and $P$ denote a target and proposal distribution supplied as input, and let $K$ be the number of samples examined by the algorithm. We show that the expected runtime $\mathbb{E}[K]$ of any CRS scales at least as $\exp_2(D_\infty[Q || P])$, where $D_\infty[Q || P]$ is the R\'enyi $\infty$-divergence. Regarding the codelength, we show that $D_{KL}[Q || P] \leq D_{CS}[Q || P] \leq \mathbb{H}[K]$, where $D_{CS}[Q || P]$ is a new quantity we call the channel simulation divergence. Furthermore, we prove that our new lower bound, unlike the $D_{KL}[Q || P]$ lower bound, is achievable tightly, i.e. there is a CRS such that $\mathbb{H}[K] \leq D_{CS}[Q || P] + \log_2 (e + 1)$. Finally, we conduct numerical studies of the asymptotic scaling of the codelength of Gaussian and Laplace channel simulation algorithms.
翻译:单次信道模拟最近作为基于机器学习的无损数据压缩方案中量化和熵编码的有前景替代方案而备受关注。然而,尽管信道模拟存在多种潜在应用(例如具有真实性约束或差分隐私的无损压缩),但对其基本局限性的了解仍然有限。本文聚焦于一类称为因果拒绝采样器(CRS)的信道模拟协议,建立了关于其预期运行时间和码长的新更紧下界,并证明了这些下界的可达性。具体而言,对于任意CRS,设$Q$和$P$分别表示作为输入提供的目标分布和建议分布,$K$为算法检查的样本数量。我们证明任何CRS的预期运行时间$\mathbb{E}[K]$至少以$\exp_2(D_\infty[Q || P])$的尺度缩放,其中$D_\infty[Q || P]$为Rényi $\infty$散度。关于码长,我们证明$D_{KL}[Q || P] \leq D_{CS}[Q || P] \leq \mathbb{H}[K]$,其中$D_{CS}[Q || P]$是称为信道模拟散度的新量。此外,我们证明与$D_{KL}[Q || P]$下界不同,新下界是可紧密达到的,即存在CRS使得$\mathbb{H}[K] \leq D_{CS}[Q || P] + \log_2 (e + 1)$。最后,我们对高斯和拉普拉斯信道模拟算法的码长渐近缩放进行了数值研究。