We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy, along with the Kullback-Leibler divergence. These measures are examined for several distributions, including gamma, chi-squared, exponential, Laplace, and log-normal distributions. We investigate the dependence of the entropy on the parameters of the respective distribution. We also study the convergence of Shannon entropy for certain probability distributions. Furthermore, we identify the extreme values of Shannon entropy for Gaussian vectors.
翻译:本文计算并分析了若干选定概率分布的多种熵测度及其性质。所考察的熵包括香农熵、Rényi熵、广义Rényi熵、Tsallis熵、Sharma-Mittal熵、修正香农熵以及Kullback-Leibler散度。这些测度在多种分布(包括伽马分布、卡方分布、指数分布、拉普拉斯分布和对数正态分布)中进行了检验。我们探究了熵对相应分布参数的依赖性,并研究了特定概率分布下香农熵的收敛特性。此外,我们确定了高斯向量香农熵的极值。