The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if \(X_1,\ldots,X_n\) are independent random variables taking values in \(\{0,1,2\}\), then the entropy of \(S_n=X_1+\cdots+X_n\) is maximized when \(X_1,\ldots,X_{n-1}\) are uniform on \(\{0,2\}\) and the probability mass function of \(X_n\) is given by \(\Prob(X_n=0) = \Prob(X_n=2) = w/2\), \(\Prob(X_n=1) = 1-w\), where \(w = \big(1 + 2^{-H(B_n)+H(B_{n-1})}\big)^{-1}\) and \(B_m\sim \Bin(m,1/2)\). The statement can be seen as an extension to ternary alphabets of the Shepp--Olkin--Mateev theorem. The proof uses the Hermite--Biehler theorem, Newton's inequalities, and Yu's maximum-entropy theorem for ultra-log-concave distributions.
翻译:考虑有限字母表上独立随机变量之和的香农熵最大化这一经典问题,本文针对三元情形予以解决。具体证明以下定理:若\(X_1,\ldots,X_n\)为取值于\(\{0,1,2\}\)的独立随机变量,则当\(X_1,\ldots,X_{n-1}\)服从\(\{0,2\}\)上的均匀分布,且\(X_n\)的概率质量函数满足\(\Prob(X_n=0) = \Prob(X_n=2) = w/2\),\(\Prob(X_n=1) = 1-w\)时,和\(S_n=X_1+\cdots+X_n\)的熵达到最大,其中\(w = \big(1 + 2^{-H(B_n)+H(B_{n-1})}\big)^{-1}\),\(B_m\sim \Bin(m,1/2)\)。该结论可视为Shepp–Olkin–Mateev定理在三元字母表上的推广。证明过程中运用了Hermite–Biehler定理、Newton不等式以及Yu关于超对数凹分布的最大熵定理。