Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its utility, the applicability of the PID is restricted by the need to assign elements as either inputs or targets, as well as the specific structure of the mutual information itself. Here, we introduce a generalized information decomposition that relaxes the source/target distinction while still satisfying the basic intuitions about information. This approach is based on the decomposition of the Kullback-Leibler divergence, and consequently allows for the analysis of any information gained when updating from an arbitrary prior to an arbitrary posterior. Consequently, any information-theoretic measure that can be written in as a Kullback-Leibler divergence admits a decomposition in the style of Williams and Beer, including the total correlation, the negentropy, and the mutual information as special cases. In this paper, we explore how the generalized information decomposition can reveal novel insights into existing measures, as well as the nature of higher-order synergies. We show that synergistic information is intimately related to the well-known Tononi-Sporns-Edelman (TSE) complexity, and that synergistic information requires a similar integration/segregation balance as a high TSE complexity. Finally, we end with a discussion of how this approach fits into other attempts to generalize the PID and the possibilities for empirical applications.

翻译：自提出以来，部分信息分解（PID）已成为一种强大的信息论技术，用于研究复杂系统中（潜在高阶）交互的结构。尽管其应用广泛，但PID的适用性受限于需要将元素指定为输入或目标，以及互信息本身的具体结构。本文提出一种广义信息分解方法，在放松源/目标区分的同时，仍满足信息的基本直觉。该方法基于Kullback-Leibler散度的分解，从而能够分析从任意先验分布更新到任意后验分布时获得的信息。因此，任何可表示为Kullback-Leibler散度的信息论度量均可按Williams和Beer的风格进行分解，包括总相关、负熵和互信息作为特例。本文探讨了广义信息分解如何揭示现有度量的新见解以及高阶协同效应的本质。我们证明，协同信息与著名的Tononi-Sporns-Edelman（TSE）复杂度密切相关，且协同信息需要与高TSE复杂度相似的整合/分离平衡。最后，我们讨论了该方法与其他泛化PID尝试的关联，以及实证应用的可能性。