Shannon's mutual information quantifies redundancy between two random variables. We introduce a new notion, termed higher-order common information (HCI), which captures the information shared among $n$ arbitrarily distributed random variables. The quantity is defined through an iterative information-bottleneck construction and can be interpreted as the maximum rate at which a single compressed representation can simultaneously preserve information about all variables. For jointly Gaussian and Bernoulli sources, we derive closed-form expressions for any $n$. We furthermore show that the HCI yields strictly tighter characterizations of redundancy than existing bounds, and demonstrate how to numerically approximate the HCI for arbitrarily distributed sources.

翻译：香农互信息量化了两个随机变量之间的冗余性。我们提出了一种新概念，称为高阶公共信息（HCI），它捕捉了$n$个任意分布随机变量之间共享的信息。该量通过迭代信息瓶颈构造定义，可解释为单个压缩表示能够同时保留所有变量信息的最大速率。对于联合高斯源和伯努利源，我们推导了任意$n$的闭式表达式。此外，我们证明HCI相比现有界能给出更严格的冗余性刻画，并展示了如何数值近似任意分布源的HCI。