We consider channel simulation protocols between two communicating parties, Alice and Bob. First, Alice receives a target distribution $Q$, unknown to Bob. Then, she employs a shared coding distribution $P$ to send the minimum amount of information to Bob so that he can simulate a single sample $X \sim Q$. For discrete distributions, Harsha et al. (2009) developed a well-known channel simulation protocol -- greedy rejection sampling (GRS) -- with a bound of ${D_{KL}[Q \,\Vert\, P] + 2\ln(D_{KL}[Q \,\Vert\, P] + 1) + \mathcal{O}(1)}$ on the expected codelength of the protocol. In this paper, we extend the definition of GRS to general probability spaces and allow it to adapt its proposal distribution after each step. We call this new procedure Adaptive GRS (AGRS) and prove its correctness. Furthermore, we prove the surprising result that the expected runtime of GRS is exactly $\exp(D_\infty[Q \,\Vert\, P])$, where $D_\infty[Q \,\Vert\, P]$ denotes the R\'enyi $\infty$-divergence. We then apply AGRS to Gaussian channel simulation problems. We show that the expected runtime of GRS is infinite when averaged over target distributions and propose a solution that trades off a slight increase in the coding cost for a finite runtime. Finally, we describe a specific instance of AGRS for 1D Gaussian channels inspired by hybrid coding. We conjecture and demonstrate empirically that the runtime of AGRS is $\mathcal{O}(D_{KL}[Q \,\Vert\, P])$ in this case.
翻译:考虑两个通信方Alice与Bob之间的信道模拟协议。首先,Alice获取一个Bob未知的目标分布$Q$。随后,她利用共享的编码分布$P$向Bob发送最少量的信息,使其能够模拟单个样本$X \sim Q$。针对离散分布,Harsha等人(2009)提出了一种著名的信道模拟协议——贪婪拒绝采样(GRS),其期望码长上界为${D_{KL}[Q \,\Vert\, P] + 2\ln(D_{KL}[Q \,\Vert\, P] + 1) + \mathcal{O}(1)}$。本文将GRS的定义推广至一般概率空间,并允许其在每一步自适应调整提议分布。我们将这一新过程称为自适应GRS(AGRS),并证明其正确性。此外,我们证明了一个令人惊讶的结果:GRS的期望运行时间恰好为$\exp(D_\infty[Q \,\Vert\, P])$,其中$D_\infty[Q \,\Vert\, P]$表示Rényi $\infty$-散度。随后,我们将AGRS应用于高斯信道模拟问题。研究表明,当对目标分布取平均时,GRS的期望运行时间为无穷大,为此我们提出一种解决方案,以略微增加编码代价为代价换取有限运行时间。最后,我们受混合编码启发,描述了针对一维高斯信道的AGRS具体实例。通过实验推测并验证,在此情况下AGRS的运行时间为$\mathcal{O}(D_{KL}[Q \,\Vert\, P])$。