Information Bottleneck (IB) is a widely used framework that enables the extraction of information related to a target random variable from a source random variable. In the objective function, IB controls the trade-off between data compression and predictiveness through the Lagrange multiplier $\beta$. Traditionally, to find the trade-off to be learned, IB requires a search for $\beta$ through multiple training cycles, which is computationally expensive. In this study, we introduce Flexible Variational Information Bottleneck (FVIB), an innovative framework for classification task that can obtain optimal models for all values of $\beta$ with single, computationally efficient training. We theoretically demonstrate that across all values of reasonable $\beta$, FVIB can simultaneously maximize an approximation of the objective function for Variational Information Bottleneck (VIB), the conventional IB method. Then we empirically show that FVIB can learn the VIB objective as effectively as VIB. Furthermore, in terms of calibration performance, FVIB outperforms other IB and calibration methods by enabling continuous optimization of $\beta$. Our codes are available at https://github.com/sotakudo/fvib.

翻译：信息瓶颈（IB）是一种广泛应用的理论框架，可从源随机变量中提取与目标随机变量相关的信息。在其目标函数中，IB通过拉格朗日乘子 $\beta$ 控制数据压缩与预测能力之间的权衡。传统上，为确定需学习的权衡参数，IB需要通过多次训练循环搜索 $\beta$，这导致计算成本高昂。本研究提出了一种创新框架——灵活变分信息瓶颈（FVIB），专门针对分类任务，能够通过单次高效训练获得所有 $\beta$ 值下的最优模型。我们从理论上证明，在所有合理 $\beta$ 值范围内，FVIB可同时最大化传统IB方法——变分信息瓶颈（VIB）目标函数的近似值。实验结果表明，FVIB学习VIB目标的效果与VIB相当。此外，在校准性能方面，FVIB通过实现 $\beta$ 的连续优化，优于其他IB方法与校准方法。我们的代码已开源至 https://github.com/sotakudo/fvib。