Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we investigate the properties of KL divergence between Gaussians. Firstly, for any two $n$-dimensional Gaussians $\mathcal{N}_1$ and $\mathcal{N}_2$, we find the supremum of $KL(\mathcal{N}_1||\mathcal{N}_2)$ when $KL(\mathcal{N}_2||\mathcal{N}_1)\leq \epsilon$ for $\epsilon>0$. This reveals the approximate symmetry of small KL divergence between Gaussians. We also find the infimum of $KL(\mathcal{N}_1||\mathcal{N}_2)$ when $KL(\mathcal{N}_2||\mathcal{N}_1)\geq M$ for $M>0$. Secondly, for any three $n$-dimensional Gaussians $\mathcal{N}_1, \mathcal{N}_2$ and $\mathcal{N}_3$, we find a tight bound of $KL(\mathcal{N}_1||\mathcal{N}_3)$ if $KL(\mathcal{N}_1||\mathcal{N}_2)$ and $KL(\mathcal{N}_2||\mathcal{N}_3)$ are bounded. This reveals that the KL divergence between Gaussians follows a relaxed triangle inequality. Importantly, all the bounds in the theorems presented in this paper are independent of the dimension $n$.

翻译：Kullback- Leiberr (KL) 的差值是概率分布中最重要的差值之一 。 在本文中, 我们调查了 Gaussians 之间 KL 差值的属性。 首先, 对于任何两个美元以上的高斯人来说, $\ mathcal{ N ⁇ 1} 和 $mathcal{N ⁇ 2} N ⁇ 1} mathcal{N ⁇ 2$ 当$( mathcal{ N_ cal{ N% 2} math cal{ N} N% 2} mathal{ N ⁇ cal{ N ⁇ 1\\\\\\\\\ leq\ leq\ epsilon$$$ $\ nleepsilon$$\ nepsilon> 。 这显示了高斯人之间小KL差值的近似比值 。 我们还发现$KL( math{ }1\\\ macal{N_\\\\\ kma) ma) 美元, 当$( mas a cal) a cal2} a cal2, 美元 美元以内 美元 美元以内 =___ *_ *_ *_ *=________mama3美元 美元 美元 美元 美元 美元 mama___\\\\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx