Popular word embedding methods such as GloVe and Word2Vec are related to the factorization of the pointwise mutual information (PMI) matrix. In this paper, we link correspondence analysis (CA) to the factorization of the PMI matrix. CA is a dimensionality reduction method that uses singular value decomposition (SVD), and we show that CA is mathematically close to the weighted factorization of the PMI matrix. In addition, we present variants of CA that turn out to be successful in the factorization of the word-context matrix, i.e. CA applied to a matrix where the entries undergo a square-root transformation (ROOT-CA) and a root-root transformation (ROOTROOT-CA). An empirical comparison among CA- and PMI-based methods shows that overall results of ROOT-CA and ROOTROOT-CA are slightly better than those of the PMI-based methods.

翻译：流行的词嵌入方法（如GloVe和Word2Vec）与点互信息（PMI）矩阵的分解密切相关。本文建立对应分析（CA）与PMI矩阵分解之间的联系。CA是一种使用奇异值分解（SVD）的降维方法，我们证明CA在数学上接近于PMI矩阵的加权分解。此外，我们提出了CA的几种变体，这些变体在词-上下文矩阵分解中表现成功，即CA应用于经过平方根变换（ROOT-CA）和根-根变换（ROOTROOT-CA）的矩阵。基于CA和基于PMI的方法之间的实证比较表明，ROOT-CA和ROOTROOT-CA的总体结果略优于基于PMI的方法。