Since its introduction in 2011, the partial information decomposition (PID) has triggered an explosion of interest in the field of multivariate information theory and the study of emergent, higher-order ("synergistic") interactions in complex systems. Despite its power, however, the PID has a number of limitations that restrict its general applicability: it scales poorly with system size and the standard approach to decomposition hinges on a definition of "redundancy", leaving synergy only vaguely defined as "that information not redundant." Other heuristic measures, such as the O-information, have been introduced, although these measures typically only provided a summary statistic of redundancy/synergy dominance, rather than direct insight into the synergy itself. To address this issue, we present an alternative decomposition that is synergy-first, scales much more gracefully than the PID, and has a straightforward interpretation. Our approach defines synergy as that information in a set that would be lost following the minimally invasive perturbation on any single element. By generalizing this idea to sets of elements, we construct a totally ordered "backbone" of partial synergy atoms that sweeps systems scales. Our approach starts with entropy, but can be generalized to the Kullback-Leibler divergence, and by extension, to the total correlation and the single-target mutual information. Finally, we show that this approach can be used to decompose higher-order interactions beyond just information theory: we demonstrate this by showing how synergistic combinations of pairwise edges in a complex network supports signal communicability and global integration. We conclude by discussing how this perspective on synergistic structure (information-based or otherwise) can deepen our understanding of part-whole relationships in complex systems.
翻译:自2011年提出以来,部分信息分解(PID)引发了多变量信息理论以及复杂系统中涌现性高阶("协同性")相互作用研究领域的兴趣爆发。然而尽管功能强大,PID仍存在若干限制其普适性的缺陷:其随系统规模扩展的效率低下,且标准分解方法依赖于对"冗余性"的定义,导致协同性仅被模糊界定为"非冗余信息"。虽然后续引入了O-信息等其他启发式度量,但这些度量通常仅提供冗余/协同主导性的汇总统计量,而非对协同性本身的直接洞见。为解决此问题,我们提出一种替代性分解方案:该方案以协同性优先为原则,扩展效率远优于PID,且具有直观解释性。我们将协同性定义为:对任意单一元素实施最小侵入性扰动后,在集合中必然丧失的那部分信息。通过将此思想推广至元素集合,我们构建了贯穿系统规模尺度的完全有序部分协同原子"骨干"结构。该方法以熵为起点,但可推广至Kullback-Leibler散度,进而延拓至总相关性和单目标互信息。最后,我们证明该方法可超越信息论范畴用于分解高阶相互作用:通过展示复杂网络中成对边的协同组合如何支持信号可通讯性与全局整合来验证其有效性。结论部分我们探讨了这种(基于信息或非信息方式的)协同结构视角如何深化对复杂系统部分-整体关系的理解。