We study the problem of exact sampling under an exponential communication cost, specifically Campbell's average codeword length $L(t)$ of order $t$, and Rényi's entropy. We provide a lower bound on the Campbell cost of exact sampling that grows approximately as $D_{1/α}(P||Q)$, the Rényi divergence of order $1/α$, with $α= \frac{1}{1+t}$. Using the Poisson functional representation of Li and El Gamal, we prove an upper bound on $L(t)$ whose leading Rényi divergence term has order within $ε$ of that of the lower bound. Our results reduce to the bounds of Harsha et al. as $α\to 1$. We also provide numerical examples comparing the bounds in the cases of normal and Laplacian distributions, demonstrating that the upper and lower bounds are typically within 5-10 bits of each other. Our results characterize exactly the optimal asymptotic Campbell cost $L(t)$ per sample as the number of independent and identically distributed (i.i.d.) samples grows to infinity. We show that under the exponential cost, any causal sampler performs strictly worse asymptotically than noncausal samplers. This contrasts with the case of expected message length, where both causal and noncausal samplers have the same optimal asymptotic cost.
翻译:我们研究了在指数通信代价下精确采样的问题,具体涉及Campbell平均码长$L(t)$(阶数为$t$)与Rényi熵。我们给出了精确采样的Campbell代价的下界,该下界近似为$D_{1/α}(P||Q)$(阶数为$1/α$的Rényi散度),其中$α= \frac{1}{1+t}$。利用Li和El Gamal的泊松泛函表示,我们证明了$L(t)$的上界,其主导Rényi散度项与下界的阶相差$ε$以内。当$α\to 1$时,我们的结果退化为Harsha等人的界。我们还提供了正态分布和拉普拉斯分布下的数值比较示例,表明上下界通常在5-10比特之间。我们的结果精确刻画了当独立同分布(i.i.d.)样本数趋于无穷时,每样本最优渐近Campbell代价$L(t)$。我们证明了在指数代价下,任何因果采样器的渐近性能严格劣于非因果采样器。这与期望消息长度的情况形成对比,其中因果和非因果采样器具有相同的最优渐近代价。