We study the information bottleneck (IB) source coding problem, also known as remote lossy source coding under logarithmic loss. Based on a rate-limited description of noisy observations, the receiver produces a soft estimate for the remote source, i.e., a probability distribution, evaluated under the logarithmic loss. We focus on the excess distortion probability of IB source coding and investigate how fast it converges to 0 or 1, depending on whether the rate is above or below the rate-distortion function. The latter case is also known as the exponential strong converse. We establish both the exact error exponent and the exact strong converse exponent for IB source coding by deriving matching upper and lower exponential bounds. The obtained exponents involve optimizations over auxiliary random variables. The matching converse bounds are derived through non-trivial extensions of existing sphere packing and single-letterization techniques, which we adapt to incorporate auxiliary random variables. In the second part of this paper, we establish a code-level connection between IB source coding and source coding with a helper, also known as the Wyner-Ahlswede-Körner (WAK) problem. We show that every code for the WAK problem is a code for IB source coding. This requires noticing that IB source coding, under the excess distortion criterion, is equivalent to source coding with a helper available at both the transmitter and the receiver; the latter in turn relates to the WAK problem. Through this connection, we re-derive the best known sphere packing exponent of the WAK problem, and provide it with an operational interpretation.
翻译:我们研究了信息瓶颈(IB)源编码问题,也称为对数损失下的远程有损源编码。接收端基于对含噪观测值的速率受限描述,生成对远程源的软估计(即概率分布),并以对数损失进行评价。我们聚焦于IB源编码的过量失真概率,探究其收敛于0或1的速度——这取决于编码速率是否超过率失真函数。后者情形也被称为指数级强逆定理。通过推导匹配的上、下指数界,我们确立了IB源编码的精确误差指数和精确强逆指数。所得指数涉及对辅助随机变量的优化。通过非平凡地推广现有球堆积和单字母化技术并纳入辅助随机变量,我们推导出匹配的逆界。在本文第二部分,我们建立了IB源编码与辅助源编码(即Wyner-Ahlswede-Körner (WAK)问题)之间在编码层面的联系。我们证明,WAK问题的每个编码都是IB源编码的一个编码。这需要注意到:在过量失真准则下,IB源编码等价于发送端和接收端均具有辅助信息的源编码;而后者又与WAK问题相关。通过这种联系,我们重新推导了WAK问题的最佳已知球堆积指数,并为其赋予操作意义上的解释。