Channel simulation algorithms can efficiently encode random samples from a prescribed target distribution $Q$ and find applications in machine learning-based lossy data compression. However, algorithms that encode exact samples usually have random runtime, limiting their applicability when a consistent encoding time is desirable. Thus, this paper considers approximate schemes with a fixed runtime instead. First, we strengthen a result of Agustsson and Theis and show that there is a class of pairs of target distribution $Q$ and coding distribution $P$, for which the runtime of any approximate scheme scales at least super-polynomially in $D_\infty[Q \Vert P]$. We then show, by contrast, that if we have access to an unnormalised Radon-Nikodym derivative $r \propto dQ/dP$ and knowledge of $D_{KL}[Q \Vert P]$, we can exploit global-bound, depth-limited A* coding to ensure $\mathrm{TV}[Q \Vert P] \leq \epsilon$ and maintain optimal coding performance with a sample complexity of only $\exp_2\big((D_{KL}[Q \Vert P] + o(1)) \big/ \epsilon\big)$.
翻译:信道模拟算法能够高效地从指定的目标分布$Q$中编码随机样本,并在基于机器学习的无损数据压缩中有应用。然而,编码精确样本的算法通常具有随机运行时间,这限制了其在需要一致编码时间场景下的适用性。因此,本文考虑固定运行时间的近似方案。首先,我们强化了Agustsson和Theis的一个结果,表明存在一类目标分布$Q$与编码分布$P$的配对,对于这些配对,任何近似方案的运行时间至少以$D_\infty[Q \Vert P]$的超多项式函数增长。随后,我们展示,相反地,如果能够访问未归一化的Radon-Nikodym导数$r \propto dQ/dP$并知道$D_{KL}[Q \Vert P]$,我们可以利用全局有界、深度受限的A*编码,确保$\mathrm{TV}[Q \Vert P] \leq \epsilon$,并在仅需$\exp_2\big((D_{KL}[Q \Vert P] + o(1)) \big/ \epsilon\big)$的样本复杂度下维持最优编码性能。