Given two jointly distributed random variables $(X,Y)$, a functional representation of $X$ is a random variable $Z$ independent of $Y$, and a deterministic function $g(\cdot, \cdot)$ such that $X=g(Y,Z)$. The problem of finding a minimum entropy functional representation is known to be equivalent to the problem of finding a minimum entropy coupling where, given a collection of probability distributions $P_1, \dots, P_m$, the goal is to find a coupling $X_1, \dots, X_m$ ($X_i \sim P_i)$ with the smallest entropy $H_\alpha(X_1, \dots, X_m)$. This paper presents a new information spectrum converse, and applies it to obtain direct lower bounds on minimum entropy in both problems. The new results improve on all known lower bounds, including previous lower bounds based on the concept of majorization. In particular, the presented proofs leverage both - the information spectrum and the majorization - perspectives on minimum entropy couplings and functional representations.
翻译:给定联合分布随机变量$(X,Y)$,$X$的函数表示是指一个与$Y$独立的随机变量$Z$及确定性函数$g(\cdot,\cdot)$,使得$X=g(Y,Z)$。最小熵函数表示问题等价于最小熵耦合问题:给定概率分布集合$P_1, \dots, P_m$,寻找耦合$X_1, \dots, X_m$($X_i \sim P_i$)使得熵$H_\alpha(X_1, \dots, X_m)$最小。本文提出一种新的信息谱反向结论,并将其应用于推导这两个问题中最小熵的直接下界。新结果改进了所有已知下界,包括基于优超概念的先验下界。特别地,本文证明方法同时利用了信息谱与优超两种视角来研究最小熵耦合与函数表示问题。