The information bottleneck (IB) problem is a widely studied framework in machine learning for extracting compressed features that are informative for downstream tasks. However, current approaches to solving the IB problem rely on a heuristic tuning of hyperparameters, offering no guarantees that the learned features satisfy information-theoretic constraints. In this work, we introduce a statistically valid solution to this problem, referred to as IB via multiple hypothesis testing (IB-MHT), which ensures that the learned features meet the IB constraints with high probability, regardless of the size of the available dataset. The proposed methodology builds on Pareto testing and learn-then-test (LTT), and it wraps around existing IB solvers to provide statistical guarantees on the IB constraints. We demonstrate the performance of IB-MHT on classical and deterministic IB formulations, validating the effectiveness of IB-MHT in outperforming conventional methods in terms of statistical robustness and reliability.

翻译：信息瓶颈（IB）问题作为机器学习中广泛研究的框架，旨在提取对下游任务具有信息量的压缩特征。然而，现有解决IB问题的方法依赖于启发式的超参数调优，无法保证所学特征满足信息论约束。本文提出了一种统计有效的解决方案，称为基于多重假设检验的信息瓶颈（IB-MHT），该方法确保所学特征以高概率满足IB约束，且与可用数据集规模无关。所提出的方法建立在帕累托检验与学习-测试（LTT）框架之上，并封装现有IB求解器以提供IB约束的统计保证。我们在经典与确定性IB公式上验证了IB-MHT的性能，结果表明其在统计鲁棒性与可靠性方面优于传统方法。