We develop an approximation method for the differential entropy $h(\mathbf{X})$ of a $q$-component Gaussian mixture in $\mathbb{R}^n$. We provide two examples of approximations using our method denoted by $\bar{h}^{\mathrm{Taylor}}_{C,m}(\mathbf{X})$ and $\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$. We show that $\bar{h}^{\mathrm{Taylor}}_{C,m}(\mathbf{X})$ provides an easy to compute lower bound to $h(\mathbf{X})$, while $\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$ provides an accurate and efficient approximation to $h(\mathbf{X})$. $\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$ is more accurate than known bounds, and conjectured to be much more resilient than the approximation of [5] in high dimensions.
翻译:我们提出了一种用于近似计算 $\mathbb{R}^n$ 空间中 $q$ 分量高斯混合分布微分熵 $h(\mathbf{X})$ 的方法。我们使用该方法给出了两个近似实例,分别记为 $\bar{h}^{\mathrm{Taylor}}_{C,m}(\mathbf{X})$ 和 $\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$。我们证明 $\bar{h}^{\mathrm{Taylor}}_{C,m}(\mathbf{X})$ 提供了一个易于计算的下界,而 $\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$ 则提供了对 $h(\mathbf{X})$ 精确且高效的近似。$\bar{h}^{\mathrm{Polyfit}}_{C,m}(\mathbf{X})$ 比已知的界更为精确,并且据推测,在高维情况下比文献[5]中的近似方法具有更强的鲁棒性。