Social bots play a significant role in many online social networks (OSN) as they imitate human behavior. This fact raises difficult questions about their capabilities and potential risks. Given the recent advances in Generative AI (GenAI), social bots are capable of producing highly realistic and complex content that mimics human creativity. As the malicious social bots emerge to deceive people with their unrealistic content, identifying them and distinguishing the content they produce has become an actual challenge for numerous social platforms. Several approaches to this problem have already been proposed in the literature, but the proposed solutions have not been widely evaluated. To address this issue, we evaluate the behavior of a text-based bot detector in a competitive environment where some scenarios are proposed: \textit{First}, the tug-of-war between a bot and a bot detector is examined. It is interesting to analyze which party is more likely to prevail and which circumstances influence these expectations. In this regard, we model the problem as a synthetic adversarial game in which a conversational bot and a bot detector are engaged in strategic online interactions. \textit{Second}, the bot detection model is evaluated under attack examples generated by a social bot; to this end, we poison the dataset with attack examples and evaluate the model performance under this condition. \textit{Finally}, to investigate the impact of the dataset, a cross-domain analysis is performed. Through our comprehensive evaluation of different categories of social bots using two benchmark datasets, we were able to demonstrate some achivement that could be utilized in future works.

翻译：社交机器人在众多在线社交网络中扮演着重要角色，因为它们模仿人类行为。这一事实引发了关于其能力和潜在风险的严峻问题。鉴于生成式人工智能（GenAI）的最新进展，社交机器人能够生成高度逼真且复杂的内容，模仿人类创造力。随着恶意社交机器人利用不真实内容欺骗用户，识别它们并区分其生成内容已成为众多社交平台面临的实际挑战。已有文献提出了多种针对该问题的方法，但这些解决方案尚未得到广泛评估。为解决此问题，我们在一个对抗性环境中评估了基于文本的机器人检测器的行为，并提出了若干场景：首先，考察机器人与机器人检测器之间的拉锯战。分析哪一方更可能获胜以及哪些因素影响这些预期，颇具意义。为此，我们将该问题建模为一个合成对抗博弈，其中对话机器人与机器人检测器进行策略性在线交互。其次，在社交机器人生成的攻击样本下评估检测模型性能；为此，我们用攻击样本污染数据集，并评估该条件下模型的表现。最后，为探究数据集影响，进行了跨域分析。通过使用两个基准数据集对不同类型的社交机器人进行全面评估，我们得以展示若干成果，这些成果可用于未来研究。