One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information is a natural means of linking variation across scales of a system due to its independence of the particular functional relationship between variables. However, estimating mutual information given high-dimensional, continuous-valued data is notoriously difficult, and the desideratum -- to reveal important variation in a comprehensible manner -- is only readily achieved through exhaustive search. Here we propose a practical, efficient, and broadly applicable methodology to decompose the information contained in a set of measurements by lossily compressing each measurement with machine learning. Guided by the distributed information bottleneck as a learning objective, the information decomposition sorts variation in the measurements of the system state by relevance to specified macroscale behavior, revealing the most important subsets of measurements for different amounts of predictive information. Additional granularity is achieved by inspection of the learned compression schemes: the variation transmitted during compression is composed of distinctions among measurement values that are most relevant to the macroscale behavior. We focus our analysis on two paradigmatic complex systems: a Boolean circuit and an amorphous material undergoing plastic deformation. In both examples, specific bits of entropy are identified out of the high entropy of the system state as most related to macroscale behavior for insight about the connection between micro- and macro- in the complex system. The identification of meaningful variation in data, with the full generality brought by information theory, is made practical for the study of complex systems.

翻译：理解复杂系统的关键步骤之一是识别系统组件尺度上的变化，这些变化与宏观尺度的行为最为相关。互信息因其不依赖于变量间特定函数关系的特性，成为连接系统不同尺度变化的自然手段。然而，针对高维连续值数据估计互信息极为困难，且其目标——以可理解的方式揭示重要变化——通常只能通过穷举搜索实现。本文提出一种实用、高效且广泛适用的方法论，通过有损压缩每个测量值（利用机器学习）来分解一组测量数据中所包含的信息。以分布式信息瓶颈作为学习目标，信息分解方法根据与指定宏观行为的相关性对系统状态的测量变化进行排序，从而揭示不同预测信息量下最重要的测量子集。通过学习到的压缩方案可进一步获得更细致的分析：压缩过程中传递的变化由与宏观行为最相关的测量值差异组成。我们聚焦于两个典型的复杂系统：布尔电路和经历塑性变形的非晶材料。在这两个例子中，从系统状态的高熵中识别出与宏观行为最相关的特定熵值，为理解复杂系统中微观与宏观之间的联系提供深刻见解。借助信息理论的普遍性，识别数据中有意义的变化变得切实可行，从而推动复杂系统的研究。