In classical information theory, a causal relationship between two variables is typically modelled by assuming that, for every possible state of one of the variables, there exists a particular distribution of states of the second variable. Let us call these two variables the causal and caused variables, respectively. We shall assume that both variables are continuous and one-dimensional. In this work we consider a procedure to transform each variable, using transformations that are differentiable and strictly increasing. We call these increasing transformations. Any causal relationship (as defined here) is associated with a channel capacity, which is the maximum rate that information could be sent if the causal relationship was used as a signalling system. Channel capacity is unaffected when the two variables are changed by use of increasing transformations. For any causal relationship we show that there is always a way to transform the caused variable such that the entropy associated with the caused variable is independent of the value of the causal variable. Furthermore, the resulting universal entropy has an absolute value that is equal to the channel capacity associated with the causal relationship. This observation may be useful in statistical applications. Also, for any causal relationship, it implies that there is a 'natural' way to transform a continuous caused variable. We also show that, with additional constraints on the causal relationship, a natural increasing transformation of both variables leads to a transformed causal relationship that has properties that might be expected from a well-engineered measuring device.
翻译:在经典信息论中,两个变量之间的因果关系通常通过假设:对于其中一个变量的每一种可能状态,都存在第二个变量的特定状态分布来建模。我们分别称这两个变量为因果变量和结果变量。我们假定这两个变量均为连续且一维的。本文考虑一种利用可微且严格递增的变换对每个变量进行变换的方法,我们称之为递增变换。任何因果关系(如本文定义)都关联一个信道容量,即若将该因果关系用作信令系统时信息传输的最大速率。当两个变量通过递增变换改变时,信道容量保持不变。对于任意因果关系,我们证明总存在一种变换结果变量的方式,使得结果变量关联的熵与因果变量的取值无关。进一步地,所得普适熵的绝对值等于该因果关系关联的信道容量。这一发现可能对统计应用有所裨益。此外,对于任意因果关系,它暗示存在一种变换连续结果变量的"自然"方式。我们还证明,在对因果关系施加额外约束的条件下,对两个变量进行自然的递增变换将得到一种变换后的因果关系,其具有可预期于精心设计的测量设备所具有的特性。