We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where $\mathsf{LProb}_{\rho}$ is category of $\ell_{\rho}$ discrete probability spaces) to $\Delta_{\mathbb{R}}$. Moreover, the Shannon entropy can be characterized as the universal monoidal natural transformation from $-/\mathsf{LProb}_{\rho}$ to the category of strongly Archimedean ordered vector spaces (a reflective subcategory of the lax-slice 2-category over $\mathsf{MonCat}_{\ell}$ in the 2-category of monoidal categories), providing a succinct characterization of Shannon entropy as a reflection arrow. We can likewise define entropy for every category with a monoidal structure on its under categories (e.g. the category of finite abelian groups, the category of finite inhabited sets, the category of finite dimensional vector spaces, and the augmented simplex category) via the reflection arrow to the reflective subcategory of strongly Archimedean ordered vector spaces. This implies that all these entropies over different categories are components of a single natural transformation (the unit of the idempotent monad), allowing us to connect these entropies in a natural manner. We also provide a universal characterization of the conditional Shannon entropy based on the chain rule which, unlike the characterization of information loss by Baez, Fritz and Leinster, does not require any continuity assumption.
翻译:我们证明了熵的本质属性(单调性、可加性和次可加性)源于熵是从下范畴函子$-/\mathsf{LProb}_{\rho}$(其中$\mathsf{LProb}_{\rho}$是$\ell_{\rho}$离散概率空间的范畴)到$\Delta_{\mathbb{R}}$的幺半自然变换。此外,香农熵可被刻画为从$-/\mathsf{LProb}_{\rho}$到强阿基米德序向量空间范畴(该范畴是幺半范畴的2-范畴中$\mathsf{MonCat}_{\ell}$上的弱切片2-范畴的一个反射子范畴)的普遍幺半自然变换,从而为香农熵提供了作为反射箭头的简洁刻画。类似地,我们可以通过指向强阿基米德序向量空间反射子范畴的反射箭头,为每个在其下范畴上具有幺半结构的范畴(例如有限阿贝尔群范畴、有限非空集合范畴、有限维向量空间范畴以及增广单纯形范畴)定义熵。这意味着这些不同范畴上的所有熵都是单一自然变换(幂等幺半群的单位元)的分量,从而允许我们以自然方式连接这些熵。我们还基于链式法则给出了条件香农熵的普适刻画,与Baez、Fritz和Leinster对信息损失的刻画不同,该刻画无需任何连续性假设。