Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation. More broadly, it highlights both the promise and risk of using LLMs to simulate online opinion dynamics.
翻译:大型语言模型越来越多地被用于模拟社交媒体用户,并推断个人在在线讨论中可能如何回应。然而,尚不明确这些模拟是否反映了精确的用户特定信念,或者它们是否对对话语境中语义无关的变化高度敏感。在本研究中,我们探讨反事实语境修订作为审核基于LLM的立场模拟的框架。给定原始在线对话,我们首先推断目标用户对特定话题的立场。然后,我们对对话语境应用受控修订策略,并在修订后的语境下再次模拟用户立场。我们比较纯文本修订策略与结合迷因语境的多元策略,并评估两个主要有效性指标,即平均方向性立场转变和立场转变率。结果显示,在文本和多元策略中,不同极化偏好机制下均存在有效且稳健的立场转变。本研究贡献了一个用于理解基于LLM的立场模拟语境敏感性的评估框架。更广泛地,它突显了使用LLM模拟在线舆论动态的前景与风险。