The information bottleneck (IB) method seeks a compressed representation of data that preserves information relevant to a target variable for prediction while discarding irrelevant information from the original data. In its classical formulation, the IB method employs mutual information to evaluate the compression between the original and compressed data and the utility of the representation for the target variable. In this study, we investigate a generalized IB problem, where the evaluation of utility is based on the $\mathcal{H}$-mutual information that satisfies the concave (\texttt{CV}) and averaging (\texttt{AVG}) conditions. This class of information measures admits a statistical decision-theoretic interpretation via its equivalence to the expected value of sample information. Based on this interpretation, we derive an alternating optimization algorithm to assess the tradeoff between compression and utility in the generalized IB problem.
翻译:信息瓶颈(IB)方法旨在寻找数据的压缩表示,该表示需保留与目标变量预测相关的信息,同时舍弃原始数据中的无关信息。在其经典表述中,IB方法使用互信息来评估原始数据与压缩数据之间的压缩程度,以及表示对目标变量的效用。本研究探讨了一个广义IB问题,其中效用的评估基于满足凹性(\texttt{CV})与平均性(\texttt{AVG})条件的$\mathcal{H}$-互信息。此类信息度量可通过其与样本信息期望值的等价性,获得一种统计决策理论的解释。基于这一解释,我们推导出一种交替优化算法,以评估广义IB问题中压缩与效用之间的权衡关系。