We consider the classical problem of learning, with arbitrary accuracy, the natural parameters of a $k$-parameter truncated \textit{minimal} exponential family from i.i.d. samples in a computationally and statistically efficient manner. We focus on the setting where the support as well as the natural parameters are appropriately bounded. While the traditional maximum likelihood estimator for this class of exponential family is consistent, asymptotically normal, and asymptotically efficient, evaluating it is computationally hard. In this work, we propose a novel loss function and a computationally efficient estimator that is consistent as well as asymptotically normal under mild conditions. We show that, at the population level, our method can be viewed as the maximum likelihood estimation of a re-parameterized distribution belonging to the same class of exponential family. Further, we show that our estimator can be interpreted as a solution to minimizing a particular Bregman score as well as an instance of minimizing the \textit{surrogate} likelihood. We also provide finite sample guarantees to achieve an error (in $\ell_2$-norm) of $\alpha$ in the parameter estimation with sample complexity $O({\sf poly}(k)/\alpha^2)$. Our method achives the order-optimal sample complexity of $O({\sf log}(k)/\alpha^2)$ when tailored for node-wise-sparse Markov random fields. Finally, we demonstrate the performance of our estimator via numerical experiments.

翻译：我们考虑经典问题：在计算和统计高效的框架下，从独立同分布样本中以任意精度学习一个$k$参数截断*极小*指数族的自然参数。我们关注支撑集与自然参数均适当有界的设定。尽管该指数族的传统最大似然估计量具有相合性、渐近正态性及渐近有效性，但其计算求解具有困难性。本文提出一种新型损失函数及一个计算高效的估计量，在温和条件下该估计量具有相合性与渐近正态性。我们证明：在总体水平上，该方法可视为对属于相同指数族分布的再参数化分布的最大似然估计。进一步，我们发现该估计量既可解释为最小化特定Bregman散度的解，也可视为最小化*代理*似然的特例。我们还提供了有限样本保障：以样本复杂度$O({\sf poly}(k)/\alpha^2)$达到参数估计误差（$\ell_2$范数）为$\alpha$。当针对节点稀疏马尔可夫随机场进行优化时，该方法实现了阶数最优的样本复杂度$O({\sf log}(k)/\alpha^2)$。最后，我们通过数值实验验证了该估计量的性能。