AI-based image editing tools, such as face swapping algorithms, can be used to transform a clothed image of a person into a sexually explicit image of that person. These tools are made easily accessible to non-expert users through mobile apps, and have been linked to reports of image-based sexual abuse and cyberbullying involving synthetic non-consensual intimate imagery. Apple and Google have begun to remove "nudification" apps from their platforms: apps that are marketed with the capability to "undress", "nudify", or create nude face swaps from images of people. However, AI image editing apps that have the same underlying capabilities, but do not present as nudification apps could be also abused to create non-consensual explicit images. In this paper, we investigate whether AI face swap apps for iOS and Android implement safety measures to prevent the creation of SNCII. We identified and downloaded 420 face swap apps, and manually tested 155 eligible apps to see whether they would permit the user to create face swaps with nude images. Our evaluation shows that 70% of apps with face swap functionality have no technical safeguards against generation of nude images. Additionally, we investigated whether face swap apps' descriptions, terms of service, or privacy policies addressed harmful uses of the app, finding that no apps self-describe as nudification apps, but that the majority do not have specific terms of service provisions prohibiting this kind of use. Our findings suggest that to mitigate the threat of UI-bound SNCII threats, platforms and lawmakers must implement policies to mandate safety filters in dual-use AI image editing applications like face swap apps.

翻译：基于AI的图像编辑工具（如换脸算法）可用于将个体的 clothed 图像转换为该个体的性暴露图像。这些工具通过移动应用向非专业用户提供便捷访问，并与涉及合成非自愿亲密图像的图像性虐待和网络霸凌报告相关联。苹果和谷歌已开始从其平台移除“裸体化”应用：那些宣称具备“脱衣”“裸体化”或基于人物图像创建裸体换脸功能的应用。然而，具备相同底层能力但不以裸体化应用形式呈现的AI图像编辑应用，同样可能被滥用以创建非自愿的暴露图像。本文系统研究了iOS和安卓平台上的AI换脸应用是否实施了防止创建非自愿合成亲密图像的安全措施。我们识别并下载了420款换脸应用，对其中155款符合条件的应用进行了人工测试，以验证其是否允许用户使用裸体图像创建换脸。评估结果显示，70%具备换脸功能的应用未设置防止生成裸体图像的技术防护措施。此外，我们考察了换脸应用描述、服务条款或隐私政策是否涉及有害使用场景，发现没有应用自我描述为裸体化应用，但大多数应用未在服务条款中明确禁止此类使用行为。我们的研究结果表明，为减轻UI导向的非自愿合成亲密图像威胁，平台和立法者必须制定政策，强制要求在双用途AI图像编辑应用（如换脸应用）中嵌入安全过滤器。