Directed information (DI) is an information measure that attempts to capture directionality in the flow of information from one random process to another. It is closely related to other causal influence measures, such as transfer entropy, Granger causality, and Pearl's causal framework. This monograph provides an overview of DI and its main application in information theory, namely, characterizing the capacity of channels with feedback and memory. We begin by reviewing the definitions of DI, its basic properties, and its relation to Shannon's mutual information. Next, we provide a survey of DI estimation techniques, ranging from classic plug-in estimators to modern neural-network-based estimators. Considering the application of channel capacity estimation, we describe how such estimators numerically optimize DI rate over a class of joint distributions on input and output processes. A significant part of the monograph is devoted to techniques to compute the feedback capacity of finite-state channels (FSCs). The feedback capacity of a strongly connected FSC involves the maximization of the DI rate from the channel input process to the output process. This maximization is performed over the class of causal conditioned probability input distributions. When the FSC is also unifilar, i.e., the next state is given by a time-invariant function of the current state and the new input-output symbol pair, the feedback capacity is the optimal average reward of an appropriately formulated Markov decision process (MDP). This MDP formulation has been exploited to develop several methods to compute exactly, or at least estimate closely, the feedback capacity of a unifilar FSC. This monograph describes these methods, starting from the value iteration algorithm, to Q-graph methods, and reinforcement learning algorithms that can handle large input and output alphabets.

翻译：定向信息是一种信息度量方法，旨在捕捉从一个随机过程到另一个随机过程的信息流向的方向性。它与其他因果影响度量密切相关，例如传递熵、格兰杰因果关系和珀尔的因果框架。本专著概述了定向信息及其在信息论中的主要应用，即刻画具有反馈和记忆的信道的容量。我们首先回顾定向信息的定义、基本性质及其与香农互信息的关系。接着，我们综述了定向信息的估计技术，从经典的插件估计器到基于神经网络的现代估计器。考虑到信道容量估计的应用，我们描述了此类估计器如何在输入与输出过程的联合分布类上数值优化定向信息率。本专著的相当一部分内容致力于计算有限状态信道反馈容量的技术。强连通有限状态信道的反馈容量涉及从信道输入过程到输出过程的定向信息率的最大化。该最大化在因果条件概率输入分布类上进行。当有限状态信道也是单义（unifilar）的，即下一状态由当前状态与新的输入-输出符号对的时不变函数给定时，反馈容量是适当构建的马尔可夫决策过程的最优平均奖励。这一马尔可夫决策过程表述已被用于开发多种方法来精确计算或至少紧密估计单义有限状态信道的反馈容量。本专著描述了这些方法，从值迭代算法，到Q图方法，再到能够处理大输入输出字母表的强化学习算法。