Beneficial to advanced computing devices, models with massive parameters are increasingly employed to extract more information to enhance the precision in describing and predicting the patterns of objective systems. This phenomenon is particularly pronounced in research domains associated with deep learning. However, investigations of causal relationships based on statistical and informational theories have posed an interesting and valuable challenge to large-scale models in the recent decade. Macroscopic models with fewer parameters can outperform their microscopic counterparts with more parameters in effectively representing the system. This valuable situation is called "Causal Emergence." This paper introduces a quantification framework, according to the Effective Information and Transition Probability Matrix, for assessing numerical conditions of Causal Emergence as theoretical constraints of its occurrence. Specifically, our results quantitatively prove the cause of Causal Emergence. By a particular coarse-graining strategy, optimizing uncertainty and asymmetry within the model's causal structure is significantly more influential than losing maximum information due to variations in model scales. Moreover, by delving into the potential exhibited by Partial Information Decomposition and Deep Learning networks in the study of Causal Emergence, we discuss potential application scenarios where our quantification framework could play a role in future investigations of Causal Emergence.
翻译:有益于先进计算设备,参数庞大的模型被日益广泛地用于提取更多信息,以增强描述和预测客观系统模式的精确性。这一现象在深度学习相关研究领域尤为显著。然而,近十年来,基于统计和信息理论的因果关联研究对大规模模型提出了一个有趣且有价值的挑战:参数更少的宏观模型在有效表征系统方面,可以优于参数更多的微观对应模型。这一有价值的情形被称为“因果涌现”。本文根据有效信息和转移概率矩阵,引入了一个量化框架,用于评估因果涌现的数理条件,作为其发生的理论约束。具体而言,我们的结果定量地证明了因果涌现的成因。通过一种特定的粗粒化策略,优化模型因果结构中的不确定性和非对称性,比因模型尺度变化而丢失最大信息具有更显著的影响力。此外,通过深入探讨部分信息分解和深度学习网络在因果涌现研究中展现的潜力,我们讨论了未来因果涌现研究中该量化框架可能发挥作用的潜在应用场景。