The Kullback-Leibler (KL) divergence is not a proper distance metric and does not satisfy the triangle inequality, posing theoretical challenges in certain practical applications. Existing work has demonstrated that KL divergence between multivariate Gaussian distributions follows a relaxed triangle inequality. Given any three multivariate Gaussian distributions $\mathcal{N}_1, \mathcal{N}_2$, and $\mathcal{N}_3$, if $KL(\mathcal{N}_1, \mathcal{N}_2)\leq ε_1$ and $KL(\mathcal{N}_2, \mathcal{N}_3)\leq ε_2$, then $KL(\mathcal{N}_1, \mathcal{N}_3)< 3ε_1+3ε_2+2\sqrt{ε_1ε_2}+o(ε_1)+o(ε_2)$. However, the supremum of $KL(\mathcal{N}_1, \mathcal{N}_3)$ is still unknown. In this paper, we investigate the relaxed triangle inequality for the KL divergence between multivariate Gaussian distributions and give the supremum of $KL(\mathcal{N}_1, \mathcal{N}_3)$ as well as the conditions when the supremum can be attained. When $ε_1$ and $ε_2$ are small, the supremum is $ε_1+ε_2+2\sqrt{ε_1ε_2}+o(ε_1)+o(ε_2)$. Finally, we demonstrate several applications of our results in out-of-distribution detection with flow-based generative models and safe reinforcement learning.

翻译：Kullback-Leibler（KL）散度并非严格的距离度量，且不满足三角不等式，这给某些实际应用带来了理论上的挑战。已有研究表明，多元高斯分布间的KL散度满足一种松弛的三角不等式。给定任意三个多元高斯分布 $\mathcal{N}_1$、$\mathcal{N}_2$ 和 $\mathcal{N}_3$，若 $KL(\mathcal{N}_1, \mathcal{N}_2)\leq ε_1$ 且 $KL(\mathcal{N}_2, \mathcal{N}_3)\leq ε_2$，则 $KL(\mathcal{N}_1, \mathcal{N}_3)< 3ε_1+3ε_2+2\sqrt{ε_1ε_2}+o(ε_1)+o(ε_2)$。然而，$KL(\mathcal{N}_1, \mathcal{N}_3)$ 的上确界仍然未知。本文研究了多元高斯分布间KL散度的松弛三角不等式，给出了 $KL(\mathcal{N}_1, \mathcal{N}_3)$ 的上确界以及该上确界可达的条件。当 $ε_1$ 和 $ε_2$ 较小时，该上确界为 $ε_1+ε_2+2\sqrt{ε_1ε_2}+o(ε_1)+o(ε_2)$。最后，我们展示了本研究结果在基于流的生成模型的分布外检测以及安全强化学习中的若干应用。