Romance-baiting scams have become a major source of financial and emotional harm worldwide. These operations are run by organized crime syndicates that traffic thousands of people into forced labor, requiring them to build emotional intimacy with victims over weeks of text conversations before pressuring them into fraudulent cryptocurrency investments. Because the scams are inherently text-based, they raise urgent questions about the role of Large Language Models (LLMs) in both current and future automation. We investigate this intersection by interviewing 145 insiders and 5 scam victims, performing a blinded long-term conversation study comparing LLM scam agents to human operators, and executing an evaluation of commercial safety filters. Our findings show that LLMs are already widely deployed within scam organizations, with 87% of scam labor consisting of systematized conversational tasks readily susceptible to automation. In a week-long study, an LLM agent not only elicited greater trust from study participants (p=0.007) but also achieved higher compliance with requests than human operators (46% vs. 18% for humans). Meanwhile, popular safety filters detected 0.0% of romance baiting dialogues. Together, these results suggest that romance-baiting scams may be amenable to full-scale LLM automation, while existing defenses remain inadequate to prevent their expansion.

翻译：情感诱饵诈骗已成为全球范围内造成重大财务与情感伤害的主要源头。此类犯罪活动由有组织犯罪集团运营，他们通过人口贩卖迫使数千人从事强迫劳动，要求这些人员在数周的文本对话中与受害者建立情感亲密关系，继而胁迫其进行欺诈性加密货币投资。由于此类诈骗本质上基于文本交流，这引发了关于大型语言模型在当前及未来自动化进程中角色的紧迫问题。我们通过访谈145名内部人员与5名诈骗受害者、开展对比LLM诈骗代理与人工操作者的双盲长期对话研究，并对商业安全过滤器进行评估，以探究这一交叉领域。研究发现：LLM已在诈骗组织中广泛部署，87%的诈骗劳动由系统化的对话任务构成，这些任务极易实现自动化；在为期一周的研究中，LLM代理不仅从研究参与者处获得了更高程度的信任（p=0.007），其请求达成率也显著高于人工操作者（46%对比人类的18%）；与此同时，主流安全过滤器对情感诱饵对话的检测率为0.0%。综合而言，这些结果表明情感诱饵诈骗可能完全适用于LLM规模化自动化，而现有防御机制尚不足以遏制其扩张态势。