Most natural and engineered information-processing systems transmit information via signals that vary in time. Computing the information transmission rate or the information encoded in the temporal characteristics of these signals, requires the mutual information between the input and output signals as a function of time, i.e. between the input and output trajectories. Yet, this is notoriously difficult because of the high-dimensional nature of the trajectory space, and all existing techniques require approximations. We present an exact Monte Carlo technique called Path Weight Sampling (PWS) that, for the first time, makes it possible to compute the mutual information between input and output trajectories for any stochastic system that is described by a master equation. The principal idea is to use the master equation to evaluate the exact conditional probability of an individual output trajectory for a given input trajectory, and average this via Monte Carlo sampling in trajectory space to obtain the mutual information. We present three variants of PWS, which all generate the trajectories using the standard stochastic simulation algorithm. While Direct PWS is a brute-force method, Rosenbluth-Rosenbluth PWS exploits the analogy between signal trajectory sampling and polymer sampling, and Thermodynamic Integration PWS is based on a reversible work calculation in trajectory space. PWS also makes it possible to compute the mutual information between input and output trajectories for systems with hidden internal states as well as systems with feedback from output to input. Applying PWS to the bacterial chemotaxis system, consisting of 182 coupled chemical reactions, demonstrates not only that the scheme is highly efficient, but also that the number of receptor clusters is much smaller than hitherto believed, while their size is much larger.

翻译：大多数自然和工程信息处理系统通过随时间变化的信号传输信息。计算信息传输速率或这些信号时间特征所编码的信息，需要输入与输出信号随时间变化的互信息，即输入与输出轨迹之间的互信息。然而，由于轨迹空间的高维特性，这一计算极为困难，且现有所有技术均需引入近似。我们提出一种名为路径权重采样（PWS）的精确蒙特卡洛技术，首次能够为任何由主方程描述的随机系统计算输入与输出轨迹之间的互信息。其核心思想是利用主方程评估给定输入轨迹下单个输出轨迹的精确条件概率，并通过轨迹空间中的蒙特卡洛采样取平均以获得互信息。我们提出了PWS的三种变体，它们均使用标准随机模拟算法生成轨迹。其中，直接PWS为暴力法，罗森布鲁斯-罗森布鲁斯PWS利用了信号轨迹采样与聚合物采样之间的类比，而热力学积分PWS基于轨迹空间中的可逆功计算。此外，PWS还可用于计算具有隐藏内部状态的系统以及输出到输入存在反馈的系统中输入与输出轨迹间的互信息。将PWS应用于由182个耦合化学反应组成的细菌趋化系统，不仅证明该方案高效，还表明受体簇的数量远小于此前认知，而其尺寸则远大于此前估计。