Iterative minimization algorithms appear in various areas including machine learning, neural networks, and information theory.The em algorithm is one of the famous iterative minimization algorithms in the area of machine learning, and the Arimoto-Blahut algorithm is a typical iterative algorithm in the area of information theory.However, these two topics had been separately studied for a long time. In this paper, we generalize an algorithm that was recently proposed in the context of the Arimoto-Blahut algorithm.Then, we show various convergence theorems, one of which covers the case when each iterative step is done approximately.Also, we apply this algorithm to the target problem of the em algorithm, and propose its improvement. In addition, we apply it to other various problems in information theory.
翻译:迭代最小化算法出现在机器学习、神经网络和信息论等多个领域。EM算法是机器学习领域最著名的迭代最小化算法之一,而Arimoto-Blahut算法则是信息论领域的典型迭代算法。然而,这两个主题长期以来一直被分别研究。本文推广了一种近期在Arimoto-Blahut算法背景下提出的算法。随后,我们展示了多种收敛定理,其中一种涵盖了每次迭代步骤近似完成的情况。此外,我们将该算法应用于EM算法的目标问题,并提出了改进方案。同时,我们还将其应用于信息论中的其他多种问题。