Information processing in networks entails a dynamical transfer of information between stochastic variables. Transfer entropy is widely used for quantification of the directional transfer of information between input and output trajectories. However, currently there is no exact technique to quantify transfer entropy given the dynamical model of a general network. Here we introduce an exact computational algorithm, Transfer Entropy-Path Weight Sampling (TE-PWS), to quantify transfer entropy and its variants in an arbitrary network in the presence of multiple hidden variables, nonlinearity, transient conditions, and feedback. TE-PWS extends a recently introduced algorithm Path Weight Sampling (PWS) and uses techniques from the statistical physics of polymers and trajectory sampling. We apply TE-PWS to linear and nonlinear systems to reveal how transfer entropy can overcome naive applications of data processing inequalities in presence of feedback.

翻译：网络中的信息处理涉及随机变量之间的动态信息传递。转移熵被广泛用于量化输入与输出轨迹之间的定向信息传递。然而，目前尚缺乏针对一般网络动力学模型的转移熵精确量化技术。本文提出一种精确计算算法——转移熵-路径权重采样（TE-PWS），用于在存在多重隐变量、非线性、瞬态条件和反馈的任意网络中量化转移熵及其变体。TE-PWS扩展了近期提出的路径权重采样（PWS）算法，并运用了聚合物统计物理与轨迹采样的技术。我们将TE-PWS应用于线性和非线性系统，揭示了在存在反馈时转移熵如何克服数据处理不等式的简单应用。