In recent several years, the information bottleneck (IB) principle provides an information-theoretic framework for deep multi-view clustering (MVC) by compressing multi-view observations while preserving the relevant information of multiple views. Although existing IB-based deep MVC methods have achieved huge success, they rely on variational approximation and distribution assumption to estimate the lower bound of mutual information, which is a notoriously hard and impractical problem in high-dimensional multi-view spaces. In this work, we propose a new differentiable information bottleneck (DIB) method, which provides a deterministic and analytical MVC solution by fitting the mutual information without the necessity of variational approximation. Specifically, we first propose to directly fit the mutual information of high-dimensional spaces by leveraging normalized kernel Gram matrix, which does not require any auxiliary neural estimator to estimate the lower bound of mutual information. Then, based on the new mutual information measurement, a deterministic multi-view neural network with analytical gradients is explicitly trained to parameterize IB principle, which derives a deterministic compression of input variables from different views. Finally, a triplet consistency discovery mechanism is devised, which is capable of mining the feature consistency, cluster consistency and joint consistency based on the deterministic and compact representations. Extensive experimental results show the superiority of our DIB method on 6 benchmarks compared with 13 state-of-the-art baselines.

翻译：近年来，信息瓶颈原理通过压缩多视图观测信息同时保留多视图相关特征，为深度多视图聚类提供了信息论框架。尽管现有基于IB的深度多视图聚类方法取得了巨大成功，但它们依赖变分近似和分布假设来估计互信息下界，这在多视图高维空间中是一个公认的棘手且不切实际的问题。本文提出一种新型可微信息瓶颈方法，通过无需变分近似的互信息拟合，提供确定性的解析多视图聚类解决方案。具体而言，我们首先利用归一化核格拉姆矩阵直接拟合高维空间互信息，无需任何辅助神经估计器来估计互信息下界。随后基于新的互信息度量，显式训练具有解析梯度的确定性多视图神经网络以参数化IB原理，从而获得来自不同视图的输入变量的确定性压缩。最后，设计了三元组一致性发现机制，能够基于确定性和紧凑表示挖掘特征一致性、聚类一致性与联合一致性。在6个基准数据集上与13个最新基线方法的对比实验表明，本DIB方法具有明显优越性。