Many online hate groups exist to disparage others based on race, gender identity, sex, or other characteristics. The accessibility of these communities allows users to join multiple types of hate groups (e.g., a racist community and misogynistic community), which calls into question whether these peripatetic users could be further radicalized compared to users that stay in one type of hate group. However, little is known about the dynamics of joining multiple types of hate groups, nor the effect of these groups on peripatetic users. In this paper, we develop a new method to classify hate subreddits, and the identities they disparage, which we use to better understand how users become peripatetic (join different types of hate subreddits). The hate classification technique utilizes human-validated LLMs to extract the protected identities attacked, if any, across 168 subreddits. We then cluster identity-attacking subreddits to discover three broad categories of hate: racist, anti-LGBTQ, and misogynistic. We show that becoming active in a user's first hate subreddit can cause them to become active in additional hate subreddits of a different category. We also find that users who join additional hate subreddits, especially of a different category, become more active in hate subreddits as a whole and develop a wider hate group lexicon. We are therefore motivated to train an AI model that we find usefully predicts the hate categories users will become active in based on post text read and written. The accuracy of this model may be partly driven by peripatetic users often using the language of hate subreddits they eventually join. Overall, these results highlight the unique risks associated with hate communities on a social media platform, as discussion of alternative targets of hate may lead users to target more protected identities.
翻译:许多在线仇恨群体以种族、性别认同、性取向或其他特征为依据贬损他人。这些社区的易访问性使得用户能够加入多种类型的仇恨群体(例如种族主义社区和厌女社区),这引发了一个问题:与仅停留在单一类型仇恨群体的用户相比,这些游走型用户是否会进一步激进化。然而,目前对于加入多种类型仇恨群体的动态过程,以及这些群体对游走型用户的影响知之甚少。本文开发了一种新的方法来对仇恨子版块及其攻击的特定身份进行分类,借此深入理解用户如何成为游走型用户(即加入不同类型的仇恨子版块)。该仇恨分类技术利用经过人工验证的大型语言模型,对168个子版块中受攻击的保护性身份(若存在)进行提取。随后,通过对身份攻击型子版块进行聚类分析,我们发现了三大类仇恨:种族主义、反LGBTQ和厌女症。研究表明,用户在首个仇恨子版块中的活跃行为可能导致其在不同类别的其他仇恨子版块中也变得活跃。我们还发现,加入更多仇恨子版块(尤其是不同类别)的用户,其在整个仇恨子版块中的活跃度会提高,并发展出更广泛的仇恨群体用语。基于此,我们训练了一个AI模型,该模型能根据用户阅读和发布的文本内容,有效预测用户将活跃于哪些仇恨类别。该模型的准确性可能部分源于游走型用户经常使用其最终加入的仇恨子版块的语言特征。总体而言,这些结果凸显了社交媒体平台上仇恨社区带来的特殊风险,因为针对其他仇恨目标的讨论可能导致用户攻击更多受保护的身份群体。