Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical perspective, viewing it as a quantitative enrichment of categories of stochastic matrices. We consider two natural monoidal structures on stochastic matrices, given by the Kronecker product and the direct sum. Our main results are complete axiomatisations of Kullback-Leibler divergence and, more generally, of Rényi divergences of arbitrary order, for each such structure. Our axiomatic theories are formulated within the framework of quantitative monoidal algebra, using a graphical language of string diagrams enriched with quantitative equations.

翻译：相对熵是概率分布间的一类基本距离度量，在概率论、统计学和机器学习中具有广泛应用。本研究从范畴论视角考察相对熵，将其视为随机矩阵范畴的量化富化。我们考虑随机矩阵上由克罗内克积与直和给出的两种自然幺半结构。主要成果是针对每种结构，分别给出了Kullback-Leibler散度以及更广义的任意阶Rényi散度的完整公理化体系。这些公理理论在量化幺半代数框架内构建，采用通过量化方程富化的弦图语言进行表述。