We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy is risk reduction from no (or partial) knowledge to full knowledge about a random variable, information is risk reduction to partial knowledge. This applies to any information measure that is based on expected loss minimization, with Shannon information and variance as prominent examples. In each case, core properties like the chain rule, non-negativity, and the relationship between information and divergence are preserved. The partition induced by an observation $Y$ encodes what it knows about the sample $ω\inΩ$. Its knowledge about another random variable $X$ is the coarser partition induced by the conditional law of $X$ given $Y$. The lattice of these distribution partitions fixes how information about $X$ decomposes into redundant, synergistic, and complementary terms. Redundancy is the risk reduced by their meet, which is Aumann's common knowledge about $X$. Synergy is the gap between their join and the distribution partition of the joint observation, which, unlike for observation partitions, can be non-zero. Complementary information further decomposes into unique contributions that depend on what is already known. Every term is the risk change along one transition between knowledge states, and none is obtained from the others by subtraction. Amounts of information do not overlap or add up like parts of a whole, contrary to common intuition and to existing decompositions. What combines are the knowledge states, through meets and joins, and information quantifies what such combinations gain.

翻译：暂无翻译