Extracting higher-order structures from multivariate data has become an area of intensive study in complex systems science, as these multipartite interactions can reveal insights into fundamental features of complex systems like emergent phenomena. Information theory provides a natural language for exploring these interactions, as it elegantly formalizes the problem of comparing ``wholes" and ``parts" using joint, conditional, and marginal entropies. A large number of distinct statistics have been developed over the years, all aiming to capture different aspects of ``higher-order" information sharing. Here, we show that three of them (the dual total correlation, S-information, and O-information) are special cases of a more general function, $Δ^{k}$ which is parameterized by a free parameter $k$. For different values of $k$, we recover different measures: $Δ^{0}$ is equal to the S-information, $Δ^{1}$ is equal to the dual total correlation, and $Δ^{2}$ is equal to the negative O-information. Generally, the $Δ^{k}$ function is arranged into a hierarchy of increasingly high-order synergies; for a given value of $k$, if $Δ^{k}>0$, then the system is dominated by interactions with order greater than $k$, while if $Δ^{k}<0$, then the system is dominated by interactions with order lower than $k$. $Δ^{k}=0$ if the system is composed entirely of synergies of order-k. Using the entropic conjugation framework, we also find that the conjugate of $Δ^{k}$, which we term $Γ^{k}$ is arranged into a similar hierarchy of increasingly high-order redundancies. These results provide new insights into the nature of both higher-order redundant and synergistic interactions, and helps unify the existing zoo of measures into a more coherent structure.

翻译：从多元数据中提取高阶结构已成为复杂系统科学中一个深入研究的领域，因为这些多部分相互作用能够揭示复杂系统（如涌现现象）基本特征的内在机制。信息论为探索这些相互作用提供了一种自然的语言，因为它通过联合熵、条件熵和边缘熵优雅地将“整体”与“部分”的比较问题形式化。多年来，人们已经发展出大量不同的统计量，均旨在捕捉“高阶”信息共享的不同方面。本文证明，其中三种统计量（对偶总相关、S-信息和O-信息）是一个更广义函数 $Δ^{k}$ 的特例，该函数由一个自由参数 $k$ 参数化。对于不同的 $k$ 值，我们可以恢复不同的度量：$Δ^{0}$ 等于 S-信息，$Δ^{1}$ 等于对偶总相关，$Δ^{2}$ 等于负的 O-信息。一般而言，$Δ^{k}$ 函数被组织成一个阶数递增的高阶协同层次结构；对于给定的 $k$ 值，若 $Δ^{k}>0$，则系统由阶数大于 $k$ 的相互作用主导；若 $Δ^{k}<0$，则系统由阶数低于 $k$ 的相互作用主导。若系统完全由 $k$ 阶协同作用构成，则 $Δ^{k}=0$。利用熵共轭框架，我们还发现 $Δ^{k}$ 的共轭量（我们称之为 $Γ^{k}$）被组织成一个类似的高阶冗余层次结构。这些结果为高阶冗余和协同相互作用的本质提供了新的见解，并有助于将现有的大量度量统一到一个更连贯的框架中。