Distributed word representation (a.k.a. word embedding) is a key focus in natural language processing (NLP). As a highly successful word embedding model, Word2Vec offers an efficient method for learning distributed word representations on large datasets. However, Word2Vec lacks consideration for distances between center and context words. We propose two novel methods, Learnable Formulated Weights (LFW) and Epoch-based Dynamic Window Size (EDWS), to incorporate distance information into two variants of Word2Vec, the Continuous Bag-of-Words (CBOW) model and the Continuous Skip-gram (Skip-gram) model. For CBOW, LFW uses a formula with learnable parameters that best reflects the relationship of influence and distance between words to calculate distance-related weights for average pooling, providing insights for future NLP text modeling research. For Skip-gram, we improve its dynamic window size strategy to introduce distance information in a more balanced way. Experiments prove the effectiveness of LFW and EDWS in enhancing Word2Vec's performance, surpassing previous state-of-the-art methods.

翻译：分布式词表示（即词嵌入）是自然语言处理（NLP）中的一个关键研究方向。作为一款极为成功的词嵌入模型，Word2Vec 提供了一种在大规模数据集上高效学习分布式词表示的方法。然而，Word2Vec 缺乏对中心词与上下文词之间距离的考量。我们提出了两种新颖方法——可学习公式化权重（LFW）和基于时期的动态窗口大小（EDWS）——将距离信息融入 Word2Vec 的两种变体中，即连续词袋（CBOW）模型和连续跳元（Skip-gram）模型。对于 CBOW，LFW 采用一个具有可学习参数的公式，该公式能够最佳地反映词之间影响力与距离的关系，以计算与距离相关的平均池化权重，为未来 NLP 文本建模研究提供思路。对于 Skip-gram，我们改进了其动态窗口大小策略，以更均衡的方式引入距离信息。实验证明，LFW 和 EDWS 能有效提升 Word2Vec 的性能，并超越了此前最先进的方法。