We derive a sharp upper bound on the relative entropy $D(P\|Q)$ in terms of the two ordered chi-squared divergences $χ^2(P\|Q)$ and $χ^2(Q\|P)$. For any $A,B>0$, the constraints $χ^2(P\|Q)\leq A$ and $χ^2(Q\|P)\leq B$ yield upper bounds on $D(P\|Q)$ and $D(Q\|P)$. Two pairs of Bernoulli distributions each attain both bounds simultaneously. As a consequence, for sequences of probability mass functions $\{P_n\}$ and $\{Q_n\}$ on a common finite or countably infinite alphabet, both $D(P_n\|Q_n)$ and $D(Q_n\|P_n)$ tend to zero whenever one of the two ordered chi-squared divergences tends to zero and the other remains uniformly bounded. Without this boundedness assumption, $D(P_n\|Q_n)$ can fail to converge to zero even when $χ^2(Q_n\|P_n)\to 0$. Finally, in contrast to the attainable upper bounds, on any finite or countably infinite alphabet containing at least three elements, the infimum of $D(P\|Q)$ under the exact constraints $χ^2(P\|Q)=A$ and $χ^2(Q\|P)=B$ is zero and is not attained.
翻译:暂无翻译