This paper highlights the developing need for quantitative modes for capturing and monitoring malicious communication in social media. There has been a deliberate "weaponization" of messaging through the use of social networks including by politically oriented entities both state sponsored and privately run. The article identifies a use of AI/ML characterization of generalized "mal-info," a broad term which includes deliberate malicious narratives similar with hate speech, which adversely impact society. A key point of the discussion is that this mal-info will dramatically increase in volume, and it will become essential for sharable quantifying tools to provide support for human expert intervention. Despite attempts to introduce moderation on major platforms like Facebook and X/Twitter, there are now established alternative social networks that offer completely unmoderated spaces. The paper presents an introduction to these platforms and the initial results of a qualitative and semi-quantitative analysis of characteristic mal-info posts. The authors perform a rudimentary text mining function for a preliminary characterization in order to evaluate the modes for better-automated monitoring. The action examines several inflammatory terms using text analysis and, importantly, discusses the use of generative algorithms by one political agent in particular, providing some examples of the potential risks to society. This latter is of grave concern, and monitoring tools must be established. This paper presents a preliminary step to selecting relevant sources and to setting a foundation for characterizing the mal-info, which must be monitored. The AI/ML methods provide a means for semi-quantitative signature capture. The impending use of "mal-GenAI" is presented.

翻译：本文强调了开发量化模式以捕获和监控社交媒体中恶意通信的日益增长的需求。通过社交网络（包括由国家资助和私人运营的政治导向实体）进行的消息传递已被蓄意“武器化”。文章提出了一种利用人工智能/机器学习对广义“恶意信息”（一个涵盖蓄意恶意叙事的广泛术语，类似于仇恨言论，对社会产生不利影响）进行特征描述的方法。讨论的一个关键点是，此类恶意信息的数量将急剧增加，因此可共享的量化工具对于支持人类专家干预将变得至关重要。尽管Facebook和X/Twitter等主要平台尝试引入内容审核，但目前已经出现了提供完全无审核空间的替代社交网络。本文介绍了这些平台，并对典型恶意信息帖子进行了定性和半定量分析的初步结果。作者执行了基本的文本挖掘功能以进行初步特征描述，从而评估实现更自动化监控的模式。该研究通过文本分析检验了若干煽动性术语，并特别讨论了某一政治行为体对生成式算法的使用，提供了其潜在社会风险的一些实例。后者值得严重关切，必须建立监控工具。本文为选择相关数据源和建立恶意信息特征描述基础迈出了初步步伐，这些恶意信息必须受到监控。人工智能/机器学习方法为半定量特征捕获提供了手段。文中还提出了即将出现的“恶意生成式人工智能”的使用前景。