We propose a novel estimator of the mutual information between two ordinal vectors $x$ and $y$. Our approach is inductive (as opposed to deductive) in that it depends on the data generating distribution solely through some nonparametric properties revealing associations in the data, and does not require having enough data to fully characterize the true joint distributions $P_{x, y}$. Specifically, our approach consists of (i) noting that $I\left(y; x\right) = I\left(u_y; u_x\right)$ where $u_y$ and $u_x$ are the copula-uniform dual representations of $y$ and $x$ (i.e. their images under the probability integral transform), and (ii) estimating the copula entropies $h\left(u_y\right)$, $h\left(u_x\right)$ and $h\left(u_y, u_x\right)$ by solving a maximum-entropy problem over the space of copula densities under a constraint of the type $\alpha_m = E\left[\phi_m(u_y, u_x)\right]$. We prove that, so long as the constraint is feasible, this problem admits a unique solution, it is in the exponential family, and it can be learned by solving a convex optimization problem. The resulting estimator, which we denote MIND, is marginal-invariant, always non-negative, unbounded for any sample size $n$, consistent, has MSE rate $O(1/n)$, and is more data-efficient than competing approaches. Beyond mutual information estimation, we illustrate that our approach may be used to mitigate mode collapse in GANs by maximizing the entropy of the copula of fake samples, a model we refer to as Copula Entropy Regularized GAN (CER-GAN).

翻译：我们建议对两个 Odinal 矢量 $xxxx 和 $y 之间的相互信息进行新的估计。 我们的方法是 $u_y; u_ xright) = I\ left(y; x\right) = left(u_y; u_ AN_right) = I\ left (u_y; u_ xxright) $ 。 我们的方法是 $y 和 $u_xx$ 的感应( 而不是扣减), 因为它取决于仅仅通过某些非参数性能显示数据在数据中的分布, 而不需要足够的数据来充分描述真实的 $\xx, 美元( y_ yright) 。 我们的方法包括 (i) left( ru_xright) $xright) 和 left( left(u_y, us_xright) $xright) $( $x) $( modemodemodeal) modeal_ demodeal_ demodeal_ demodeal_ demodeal_ demodeal_ demodeal) modeal_ demodemodeal_ demodeal modeal_ modeal_ modeal_ modeal_ modeal a dismodeal_ modeal_ modemodeal_ dismodal a modeal modeal modemodeal modeal modeal modeal modeal modeal_ modeal modeal_ modeal modeal modal modeal modal modeal modal modal modal modal modal modal modal modal modal mods) mod mod mod mods) mod mods mods mods mods modal modal mod mod modal modal mo