The information bottleneck (IB) approach, initially introduced by Tishby et al. to assess the "compression--relevance" tradeoff for a remote source coding problem in communications, gains popularity recently in its application to modern machine learning (ML). Despite its seemingly simple form, the solution to IB problem remains largely unknown, and can only be assessed numerically even in the simple setting of Gaussian mixture model that is of fundamental significance in ML. In this paper, by combining ideas of hard quantization and soft nonlinear transformation, we derive closed-form achievable bounds for the IB problem under the above setting. The derived bounds establish surprisingly close behavior to the (numerically) optimal IB solution obtained by Blahut--Arimoto (BA) algorithm, on both synthetic and real-world (so non-Gaussian mixture) datasets, suggesting possibly wider applicability of our results.

翻译：信息瓶颈（IB）方法最初由Tishby等人提出，用于评估通信中远程信源编码问题的“压缩-相关性”权衡，近年来在现代机器学习（ML）应用中日益流行。尽管其形式看似简单，但IB问题的解在很大程度上仍属未知，即便是在机器学习中具有基础重要性的高斯混合模型这一简单设定下，也只能通过数值方法进行评估。本文通过结合硬量化与软非线性变换的思想，推导了上述设定下IB问题的闭式可达界。在合成数据集和真实世界（即非高斯混合）数据集上，所得界与通过Blahut-Arimoto（BA）算法获得的（数值）最优IB解表现出惊人的接近行为，这表明我们的结果可能具有更广泛的适用性。