Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its utility, the applicability of the PID is restricted by the need to assign elements as either inputs or targets, as well as the specific structure of the mutual information itself. Here, we introduce a generalized information decomposition that relaxes the source/target distinction while still satisfying the basic intuitions about information. This approach is based on the decomposition of the Kullback-Leibler divergence, and consequently allows for the analysis of any information gained when updating from an arbitrary prior to an arbitrary posterior. Consequently, any information-theoretic measure that can be written in as a Kullback-Leibler divergence admits a decomposition in the style of Williams and Beer, including the total correlation, the negentropy, and the mutual information as special cases. In this paper, we explore how the generalized information decomposition can reveal novel insights into existing measures, as well as the nature of higher-order synergies. We show that synergistic information is intimately related to the well-known Tononi-Sporns-Edelman (TSE) complexity, and that synergistic information requires a similar integration/segregation balance as a high TSE complexity. Finally, we end with a discussion of how this approach fits into other attempts to generalize the PID and the possibilities for empirical applications.

翻译：自提出以来，部分信息分解（PID）已成为一种强大的信息论技术，可用于研究复杂系统中（潜在高阶）相互作用的结构。尽管其具有实用性，但PID的应用受到以下限制：需要将元素指定为输入或目标，以及互信息本身的结构特性。本文提出了一种广义信息分解方法，它放宽了源/目标区分，同时仍满足信息的基本直觉。该方法基于Kullback-Leibler散度的分解，因此能够分析从任意先验分布更新到任意后验分布时获得的任意信息。因此，任何可以写为Kullback-Leibler散度的信息论度量都可以按Williams和Beer的风格进行分解，包括总相关、负熵和互信息作为特例。本文探讨了广义信息分解如何揭示现有度量的新见解，以及高阶协同的本质。我们展示了协同信息与著名的Tononi-Sporns-Edelman（TSE）复杂度密切相关，并且协同信息需要与高TSE复杂度类似的整合/分离平衡。最后，我们讨论了这一方法如何与其他泛化PID的尝试相契合，以及其实证应用的可能性。