Expanding a dictionary of pre-selected keywords is crucial for tasks in information retrieval, such as database query and online data collection. Here we propose Local Graph-based Dictionary Expansion (LGDE), a method that uses tools from manifold learning and network science for the data-driven discovery of keywords starting from a seed dictionary. At the heart of LGDE lies the creation of a word similarity graph derived from word embeddings and the application of local community detection based on graph diffusion to discover semantic neighbourhoods of pre-defined seed keywords. The diffusion in the local graph manifold allows the exploration of the complex nonlinear geometry of word embeddings and can capture word similarities based on paths of semantic association. We validate our method on a corpus of hate speech-related posts from Reddit and Gab and show that LGDE enriches the list of keywords and achieves significantly better performance than threshold methods based on direct word similarities. We further demonstrate the potential of our method through a real-world use case from communication science, where LGDE is evaluated quantitatively on data collected and analysed by domain experts by expanding a conspiracy-related dictionary.
翻译:扩展预选关键词词典对信息检索任务(如数据库查询和在线数据收集)至关重要。本文提出基于局部图的词典扩展方法(LGDE),该方法利用流形学习和网络科学工具,从种子词典出发进行数据驱动的关键词发现。LGDE的核心在于构建基于词嵌入的词相似图,并应用基于图扩散的局部社区检测技术,以发现预定义种子词语的语义邻域。局部图流形上的扩散机制能够探索词嵌入中复杂的非线性几何结构,并通过语义关联路径捕捉词语相似性。我们在来自Reddit和Gab平台的仇恨言论相关帖子语料库上验证该方法,结果表明LGDE能够丰富关键词列表,且性能显著优于基于直接词相似度的阈值方法。我们进一步通过传播学领域的真实用例展示了该方法的潜力:在该用例中,领域专家通过扩展阴谋论相关词典收集并分析数据,对LGDE进行了定量评估。