Recent advancements in generative artificial intelligence (AI) have raised concerns about their potential to create convincing fake social media accounts, but empirical evidence is lacking. In this paper, we present a systematic analysis of Twitter(X) accounts using human faces generated by Generative Adversarial Networks (GANs) for their profile pictures. We present a dataset of 1,353 such accounts and show that they are used to spread scams, spam, and amplify coordinated messages, among other inauthentic activities. Leveraging a feature of GAN-generated faces -- consistent eye placement -- and supplementing it with human annotation, we devise an effective method for identifying GAN-generated profiles in the wild. Applying this method to a random sample of active Twitter users, we estimate a lower bound for the prevalence of profiles using GAN-generated faces between 0.021% and 0.044% -- around 10K daily active accounts. These findings underscore the emerging threats posed by multimodal generative AI. We release the source code of our detection method and the data we collect to facilitate further investigation. Additionally, we provide practical heuristics to assist social media users in recognizing such accounts.

翻译：生成式人工智能的最新进展引发了对其可能制造令人信服的虚假社交媒体账户的担忧，但尚缺乏实证证据。本文系统分析了使用生成对抗网络生成的人脸作为头像的Twitter(X)账户。我们构建了一个包含1,353个此类账户的数据集，并证明这些账户被用于传播诈骗、垃圾信息及放大协调性消息等不真实活动。利用GAN生成人脸的特征——眼位一致性——并结合人工标注，我们设计了一种有效识别自然环境中GAN生成资料的方法。将该方法应用于随机抽样的活跃Twitter用户后，我们估计使用GAN生成人脸的资料流行率下限为0.021%至0.044%——约每日1万个活跃账户。这些发现凸显了多模态生成式AI带来的新兴威胁。我们公开了检测方法的源代码及所收集的数据，以促进后续研究。此外，我们提供了实用启发式规则，帮助社交媒体用户识别此类账户。