The task of infomin learning aims to learn a representation with high utility while being uninformative about a specified target, with the latter achieved by minimising the mutual information between the representation and the target. It has broad applications, ranging from training fair prediction models against protected attributes, to unsupervised learning with disentangled representations. Recent works on infomin learning mainly use adversarial training, which involves training a neural network to estimate mutual information or its proxy and thus is slow and difficult to optimise. Drawing on recent advances in slicing techniques, we propose a new infomin learning approach, which uses a novel proxy metric to mutual information. We further derive an accurate and analytically computable approximation to this proxy metric, thereby removing the need of constructing neural network-based mutual information estimators. Experiments on algorithmic fairness, disentangled representation learning and domain adaptation verify that our method can effectively remove unwanted information with limited time budget.

翻译：信息最小化学习的目标是学习一个具有高实用性的表示，同时避免其携带关于指定目标的信息，后者通过最小化表示与目标之间的互信息来实现。该方法应用广泛，涵盖针对保护属性训练公平预测模型，以及通过解耦表示进行无监督学习等领域。近期关于信息最小化学习的研究主要采用对抗训练，即训练神经网络来估计互信息或其代理，因此该方法速度慢且难以优化。借鉴切片技术的最新进展，我们提出了一种新的信息最小化学习方法，该方法采用一种新颖的互信息代理度量。我们进一步推导出该代理度量的精确且可解析计算的近似值，从而消除了构建基于神经网络的互信息估计器的需求。在算法公平性、解耦表示学习和领域自适应上的实验证实，我们的方法能在有限的时间预算内有效消除不必要的信息。