Hidden Markov Models (HMM) model a sequence of observations that are dependent on a hidden (or latent) state that follow a Markov chain. These models are widely used in diverse fields including ecology, speech recognition, and genetics.Parameter estimation in HMM is typically performed using the Baum-Welch algorithm, a special case of the Expectation-Maximisation (EM) algorithm. While this method guarantee the convergence to a local maximum, its convergence rates is usually slow.Alternative methods, such as the direct maximisation of the likelihood using quasi-Newton methods (such as L-BFGS-B) can offer faster convergence but can be more complicated to implement due to challenges to deal with the presence of bounds on the space of parameters.We propose a novel hybrid algorithm, QNEM, that combines the Baum-Welch and the quasi-Newton algorithms. QNEM aims to leverage the strength of both algorithms by switching from one method to the other based on the convexity of the likelihood function.We conducted a comparative analysis between QNEM, the Baum-Welch algorithm, an EM acceleration algorithm called SQUAREM (Varadhan, 2008, Scand J Statist), and the L-BFGS-B quasi-Newton method by applying these algorithms to four examples built on different models. We estimated the parameters of each model using the different algorithms and evaluated their performances.Our results show that the best-performing algorithm depends on the model considered. QNEM performs well overall, always being faster or equivalent to L-BFGS-B. The Baum-Welch and SQUAREM algorithms are faster than the quasi-Newton and QNEM algorithms in certain scenarios with multiple optimum. In conclusion, QNEM offers a promising alternative to existing algorithms.
翻译:隐马尔可夫模型(HMM)通过对依赖于遵循马尔可夫链的隐藏(或潜在)状态的观测序列进行建模,广泛应用于生态学、语音识别和遗传学等多个领域。HMM的参数估计通常采用Baum-Welch算法——期望最大化(EM)算法的一种特例。该方法虽能保证收敛到局部极大值,但其收敛速度通常较慢。替代方法(如使用拟牛顿法(例如L-BFGS-B)直接最大化似然函数)可提供更快的收敛速度,但由于参数空间存在边界约束,实现起来更为复杂。本文提出了一种结合Baum-Welch算法与拟牛顿法的新型混合算法QNEM。该算法基于似然函数的凸性在两种方法间切换,旨在综合利用两种算法的优势。我们通过将QNEM、Baum-Welch算法、EM加速算法SQUAREM(Varadhan, 2008, Scand J Statist)以及L-BFGS-B拟牛顿法应用于四个基于不同模型的实例进行比较分析,使用不同算法估计各模型参数并评估其性能。结果表明,最佳算法取决于具体模型。QNEM整体表现良好,其速度始终快于或等同于L-BFGS-B。在存在多个最优解的场景中,Baum-Welch和SQUAREM算法比拟牛顿法和QNEM算法更快。综上,QNEM为现有算法提供了一种具有前景的替代方案。