Information Bottleneck (IB) is a technique to extract information about one target random variable through another relevant random variable. This technique has garnered significant interest due to its broad applications in information theory and deep learning. Hence, there is a strong motivation to develop efficient numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a semi-relaxed IB model, where the Markov chain and transition probability condition are relaxed from the relevance-compression function. Based on the proposed model, we develop an algorithm, which recovers the relaxed constraints and involves only closed-form iterations. Specifically, the algorithm is obtained by analyzing the Lagrangian of the relaxed model with alternating minimization in each direction. The convergence property of the proposed algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical experiments across classical and discrete distributions corroborate the analysis. Moreover, our proposed algorithm demonstrates notable advantages in terms of computational efficiency, evidenced by significantly reduced run times compared to existing methods with comparable accuracy.

翻译：信息瓶颈（IB）是一种通过相关随机变量提取另一个目标随机变量信息的技术。该技术因其在信息论与深度学习中的广泛应用而备受关注。因此，开发具有高精度和理论收敛保证的高效数值方法具有重要动机。本文提出一种半松弛IB模型，其中马尔可夫链和转移概率条件从相关-压缩函数中松弛。基于该模型，我们开发了一种算法，该算法可恢复松弛约束并仅涉及闭式迭代。具体而言，该算法通过分析松弛模型的拉格朗日函数并沿各方向进行交替最小化得到。通过下降估计和平斯克尔不等式，理论上保证了所提算法的收敛性。经典分布和离散分布的数值实验验证了分析结果。此外，我们提出的算法在计算效率方面表现出显著优势，与现有方法相比，在保持相近精度的同时大幅缩短了运行时间。