The Symmetric Information Bottleneck (SIB), an extension of the more familiar Information Bottleneck, is a dimensionality reduction technique that simultaneously compresses two random variables to preserve information between their compressed versions. We introduce the Generalized Symmetric Information Bottleneck (GSIB), which explores different functional forms of the cost of such simultaneous reduction. We then explore the dataset size requirements of such simultaneous compression. We do this by deriving bounds and root-mean-squared estimates of statistical fluctuations of the involved loss functions. We show that, in typical situations, the simultaneous GSIB compression requires qualitatively less data to achieve the same errors compared to compressing variables one at a time. We suggest that this is an example of a more general principle that simultaneous compression is more data efficient than independent compression of each of the input variables.

翻译：对称信息瓶颈（Symmetric Information Bottleneck, SIB）是更具普适性的信息瓶颈方法的扩展，其作为一种降维技术，在同时压缩两个随机变量的过程中保持其压缩版本之间的互信息。本文引入广义对称信息瓶颈（Generalized Symmetric Information Bottleneck, GSIB），探讨了这种同步压缩过程中不同损失函数形式的影响。我们进一步研究了此类同步压缩所需的数据集规模，通过推导相关损失函数统计波动的界及均方根误差估计，证明在典型情形下，与逐变量压缩相比，同步GSIB压缩在达成相同误差水平时所需数据量显著更少。这一现象印证了更一般的原理：同步压缩相较于对各输入变量进行独立压缩具有更高的数据效率。