Weighted finite automata (WFA) are often used to represent probabilistic models, such as $n$-gram language models, since they are efficient for recognition tasks in time and space. The probabilistic source to be represented as a WFA, however, may come in many forms. Given a generic probabilistic model over sequences, we propose an algorithm to approximate it as a weighted finite automaton such that the Kullback-Leiber divergence between the source model and the WFA target model is minimized. The proposed algorithm involves a counting step and a difference of convex optimization step, both of which can be performed efficiently. We demonstrate the usefulness of our approach on various tasks, including distilling $n$-gram models from neural models, building compact language models, and building open-vocabulary character models. The algorithms used for these experiments are available in an open-source software library.

翻译：加权自定义模型(WFA)常常用来代表概率模型,如美元克语言模型,因为它们在时间和空间上对识别任务有效。不过,作为WFA代表的概率源可能以多种形式出现。考虑到对序列的通用概率模型,我们提议一种算法,把它作为加权自定义模型,这样可以最大限度地缩小源模型与WFA目标模型之间的 Kullback-Leiber差异。提议的算法涉及一个计数步骤和锥形优化步骤的差异,两者都可以有效完成。我们展示了我们在各种任务上的做法的效用,包括从神经模型中蒸馏一美元-克模型,建立紧凑语言模型,以及建立开放词汇字符模型。用于这些实验的算法可以在开放源软件库中查阅。