The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as $SD^2$. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.

翻译：解耦表示学习的进展通过实现对工具变量、混杂因子和可调节变量的精确控制，显著提升了反事实预测的准确性。实现这些因素独立分离的一种有效方法是互信息最小化，这一任务在许多机器学习场景中，尤其是在高维空间中，面临着挑战。为规避这一挑战，我们提出了自蒸馏解耦框架，简称为 $SD^2$。该框架基于信息论，无需为高维表示设计复杂的互信息估计器，即可确保理论上可靠的独立解耦表示。我们在合成数据集和真实数据集上进行的全面实验证实了该方法在存在观测和未观测混杂因子的情况下，促进反事实推理的有效性。