Word embedding has become ubiquitous and is widely used in various text mining and natural language processing (NLP) tasks, such as information retrieval, semantic analysis, and machine translation, among many others. Unfortunately, it is prohibitively expensive to train the word embedding in a relatively large corpus. We propose a graph-based word embedding algorithm, called Word-Graph2vec, which converts the large corpus into a word co-occurrence graph, then takes the word sequence samples from this graph by randomly traveling and trains the word embedding on this sampling corpus in the end. We posit that because of the stable vocabulary, relative idioms, and fixed expressions in English, the size and density of the word co-occurrence graph change slightly with the increase in the training corpus. So that Word-Graph2vec has stable runtime on the large scale data set, and its performance advantage becomes more and more obvious with the growth of the training corpus. Extensive experiments conducted on real-world datasets show that the proposed algorithm outperforms traditional Skip-Gram by four-five times in terms of efficiency, while the error generated by the random walk sampling is small.

翻译：词嵌入已成为文本挖掘和自然语言处理（NLP）任务中广泛应用的基础技术，包括信息检索、语义分析和机器翻译等众多领域。然而，在相对庞大的语料库上训练词嵌入的成本过高。我们提出一种基于图的词嵌入算法，名为Word-Graph2vec。该算法首先将大规模语料库转换为词共现图，然后通过随机游走从该图中采样词序列，最后在此采样语料库上训练词嵌入。我们假设，由于英语中稳定的词汇、惯用语和固定表达，词共现图的规模和密度会随着训练语料库的增加而发生微小变化。因此，Word-Graph2vec在大规模数据集上具有稳定的运行时间，且其性能优势随着训练语料库的增长而愈发显著。在真实数据集上进行的大量实验表明，该算法在效率上比传统Skip-Gram模型提高四到五倍，同时随机游走采样产生的误差较小。